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Preface 


This book will prepare you for quantitative finance interviews by helping you zero in on 
the key concepts that are frequently tested in such interviews. In this book we analyze 
solutions to more than 200 real interview problems and provide valuable insights into 
how to ace quantitative interviews. The book covers a variety of topics that you are 
likely to encounter in quantitative interviews: brain teasers, calculus, linear algebra, 
probability, stochastic processes and stochastic calculus, finance and programming. 


Professionals and students seeking to pursue a career in quantitative finance or related 
quantitative fields will benefit most from thoroughly reading this book. In recent years, 
we have seen a dramatic surge in demand for talents with strong quantitative skills from 
investment banks, investment management firms, hedge funds, financial software 
vendors and financial consulting companies. As a result, quant, an umbrella description 
that encompasses quantitative analysts, quantitative researchers, quantitative strategists, 
quantitative traders, and quantitative developers, has become an attractive career choice. 


Dozens of financial engineering or computational finance programs have been 
established in the last few years to educate professionals for quantitative finance jobs. 
Graduates with backgrounds in finance, mathematics, physics, computer sciences, and 
various engineering majors are contending for quant jobs as well. Naturally, the 
competition is fierce. To be a successful candidate, you have to distinguish yourself 
from many other excellent applicants. 


In general, a successful candidate for a quantitative finance position 1s expected to have 
a strong mathematics background (in probability, statistics, stochastic calculus, etc.), 
solid programming skills and basic to intermediate-level finance knowledge. Most 
candidates find quantitative interviews, or at least some interview problems, challenging. 


Quantitative interviews cover a broad range of mathematics, finance and programming 
topics that the candidates may have never used or even encountered in their daily work 
or study. Moreover, most interview problems require strong problem-solving skills, 
beyond reciting formulas or doing simple calculations. A successful candidate needs a 
combination of knowledge and problem-solving skills in order to excel in quantitative 
interviews. This is precisely what this book provides! 


This book addresses these aspects by reviewing the necessary finance and mathematical 
concepts that serve as tools to structure and solve interview problems. Since it includes 
most of the topics used by quantitative interviewers, it presupposes some basic 
preparation in mathematics, statistics, finance, and programming. 


I also strongly recommend that you try to solve each problem on your own first before 
reading the answer. Working out solutions on your own will help you improve your 
problem-solving skills and help you quickly identify common approaches to tackling 
quantitative problems. 


Needless to say, you are likely to encounter some problems in interviews that are similar 
to or exactly the same as the problems in this book. After all, the book covers many 
essential quantitative topics using real interview problems. However, the goal of the 
book is not to teach you how to game the system by remembering the answers! In fact, 
just memorizing answers may not help much in your interview process. Unless you truly 
understand the underlying concepts and can analyze the problems yourself, you will fail 
to elaborate on the solutions and will be ill-equipped to answer many other problems 
that use similar concepts. (Besides, many experienced quantitative interviewers are good 
at catching those who have simply memorized “canned” answers.) 


This is exactly the reason why I make significant effort to review essential concepts, to 
present solution strategies, and to analyze the solutions in detail instead of simply 
providing answers to problems. Furthermore, although the building blocks can be 
learned, how one analyzes problems and implements these concepts usually makes a big 
difference—and these are the skills you can acquire through practice, practice and 
practice. 


I realize that there may be better methods to solve some of the problems presented in 
this book. It is entirely possible that despite my best efforts some inadvertent errors may 
have crept in. Please email me at xinfeng@quantfinanceinterviews.com if you have a 
better approach to solving some of these problems or find errors. I will be grateful for 
your feedback and will post corrections and your constructive feedback on the book’s 
companion website http://www.quantfinanceinterviews.com. The website is a joint 
venture with my editor, Brett Jiu. You will also find some extra interview problems 
with answers that we have gathered. 


I sincerely hope that you enjoy solving these problems and are successful in your 
interviews. 


Xinfeng Zhou 
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Chapter 1 General Principles 


Let us begin this book by exploring five general principles that will be extremely helpful 
in your interview process. From my experience on both sides of the interview table, 
these general guidelines will better prepare you for job interviews and will likely make 
you a successful candidate. 


1. Build a broad knowledge base 


The length and the style of quant interviews differ from firm to firm. Landing a quant 
job may mean enduring hours of bombardment with brain teaser, calculus, linear algebra, 
probability theory, statistics, derivative pricing, or programming problems. To be a 
successful candidate, you need to have broad knowledge in mathematics, finance and 
programming. 


Will all these topics be relevant for your future quant job? Probably not. Each specific 
quant position often requires only limited knowledge in these domains. General problem 
solving skills may make more difference than specific knowledge. Then why are 
quantitative interviews so comprehensive? There are at least two reasons for this: 


The first reason is that interviewers often have diverse backgrounds. Each interviewer 
has his or her own favorite topics that are often related to his or her own educational 
background or work experience. As a result, the topics you will be tested on are likely to 
be very broad. The second reason is more fundamental. Your problem solving skills—a 
crucial requirement for any quant job—is often positively correlated to the breadth of 
your knowledge. A basic understanding of a broad range of topics often helps you better 
analyze problems, explore alternative approaches, and come up with efficient solutions. 
Besides, your responsibility may not be restricted to your own projects. You will be 
expected to contribute as a member of a bigger team. Having broad knowledge will help 
you contribute to the team's success as well. 


The key here is “basic understanding." Interviewers do not expect you to be an expert on 
a specific subject—unless it happens to be your PhD thesis. The knowledge used in 
interviews, although broad, covers mainly essential concepts. This is exactly the reason 
why most of the books I refer to in the following chapters have the word “introduction” 
or "first" in the title. If I am allowed to give only one suggestion to a candidate, it will be 


know the basics very well. 


2. Practice your interview skills 


The interview process starts long before you step into an interview room. In a sense, the 
success or failure of your interview is often determined before the first question is asked. 
Your solutions to interview problems may fail to reflect your true intelligence and 


General Principles 


knowledge if you are unprepared. Although a complete review of quant interview 
problems is impossible and unnecessary, practice does improve your interview skills. 
Furthermore, many of the behavioral, technical and resume-related questions can be 
anticipated. So prepare yourself for potential questions long before you enter an 
interview room. 


3. Listen carefully 


You should be an active listener in interviews so that you understand the problems well 
before you attempt to answer them. If any aspect of a problem is not clear to you, 
politely ask for clarification. If the problem is more than a couple of sentences, jot down 
the key words to help you remember all the information. For complex problems, 
interviewers often give away some clues when they explain the problem. Even the 
assumptions they give may include some information as to how to approach the problem. 
So listen carefully and make sure you get the necessary information. 


4. Speak your mind 


When you analyze a problem and explore different ways to solve it, never do it silently. 
Clearly demonstrate your analysis and write down the important steps involved if 
necessary. This conveys your intelligence to the interviewer and shows that you are 
methodical and thorough. In case that you go astray, the interaction will also give your 
interviewer the opportunity to correct the course and provide you with some hints. 


Speaking your mind does not mean explaining every tiny detail. If some conclusions are 
obvious to you, simply state the conclusion without the trivial details. More often than 
not, the interviewer uses a problem to test a specific concept/approach. You should focus 
on demonstrating your understanding of the key concept/approach instead of dwelling 
on less relevant details. 


5. Make reasonable assumptions 


In real job settings, you are unlikely to have all the necessary information or data you’d 
prefer to have before you build a model and make a decision. In interviews, 
interviewers may not give you all the necessary assumptions either. So it is up to you to 
make reasonable assumptions. The keyword here is reasonable. Explain your 
assumptions to the interviewer so that you will get immediate feedback. To solve 
quantitative problems, it is crucial that you can quickly make reasonable assumptions 
and design appropriate frameworks to solve problems based on the assumptions. 


We are now ready to review basic concepts in quantitative finance subject areas and 
have fun solving real-world interview problems! 


Chapter 2 Brain Teasers 


In this chapter, we cover problems that only require common sense, logic, reasoning, and 
basic—no more than high school level—math knowledge to solve. In a sense, they are 
real brain teasers as opposed to mathematical problems in disguise. Although these brain 
teasers do not require specific math knowledge, they are no less difficult than other 
quantitative interview problems. Some of these problems test your analytical and general 
problem-solving skills; some require you to think out of the box; while others ask you to 
solve the problems using fundamental math techniques in a creative way. In this chapter, 
we review some interview problems to explain the general themes of brain teasers that 
you are likely to encounter in quantitative interviews. 


2.1 Problem Simplification 


If the original problem is so complex that you cannot come up with an immediate 
solution, try to identify a simplified version of the problem and start with it. Usually you 
can start with the simplest sub-problem and gradually increase the complexity. You do 
not need to have a defined plan at the beginning. Just try to solve the simplest cases and 
analyze your reasoning. More often than not, you will find a pattern that will guide you 
through the whole problem. 


Screwy pirates 


Five pirates looted a chest full of 100 gold coins. Being a bunch of democratic pirates, 
they agree on the following method to divide the loot: 


The most senior pirate will propose a distribution of the coins. All pirates, including the 
most senior pirate, will then vote. If at least 50% of the pirates (3 pirates in this case) 
accept the proposal, the gold is divided as proposed. If not, the most senior pirate will be 
fed to shark and the process starts over with the next most senior pirate... The process is 
repeated until a plan is approved. You can assume that all pirates are perfectly rational: 
they want to stay alive first and to get as much gold as possible second. Finally, being 
blood-thirsty pirates, they want to have fewer pirates on the boat if given a choice 
between otherwise equal outcomes. 


How will the gold coins be divided in the end? 


Solution: If you have not studied game theory or dynamic programming, this strategy 
problem may appear to be daunting. If the problem with 5 pirates seems complex, we 
can always start with a simplified version of the problem by reducing the number of 
pirates. Since the solution to 1-pirate case is trivial, let's start with 2 pirates. The senior 
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pirate (labeled as 2) can claim all the gold since he will always get 50% of the votes 
from himself and pirate 1 is left with nothing. 


Let’s add a more senior pirate, 3. He knows that if his plan is voted down, pirate 1 will 
get nothing. But if he offers private 1 nothing, pirate 1 will be happy to kill him. So 
pirate 3 will offer private 1 one coin and keep the remaining 99 coins, in which strategy 
the plan will have 2 votes from pirate 1 and 3. 


If pirate 4 is added, he knows that if his plan is voted down, pirate 2 will get nothing. So 
pirate 2 will settle for one coin if pirate 4 offers one. So pirate 4 should offer pirate 2 one 
coin and keep the remaining 99 coins and his plan will be approved with 50% of the 
votes from pirate 2 and 4. 


Now we finally come to the 5-pirate case. He knows that if his plan is voted down, both 
pirate 3 and pirate 1 will get nothing. So he only needs to offer pirate 1 and pirate 3 one 
coin each to get their votes and keep the remaining 98 coins. If he divides the coins this 
way, he will have three out of the five votes: from pirates 1 and 3 as well as himself. 


Once we start with a simplified version and add complexity to it, the answer becomes 
obvious. Actually after the case n = 5, a clear pattern has emerged and we do not need to 


stop at 5 pirates. For any 2141 pirate case (n should be less than 99 though), the most 
senior pirate will offer pirates 1, 3, ---, and 25-1 each one coin and keep the rest for 


himself. 


Tiger and sheep 


One hundred tigers and one sheep are put on a magic island that only has grass. Tigers 
can eat grass, but they would rather eat sheep. Assume: A. Each time only one tiger can 
eat one sheep, and that tiger itself will become a sheep after it eats the sheep. B. All 
tigers are smart and perfectly rational and they want to survive. So will the sheep be 
eaten? 


Solution: 100 is a large number, so again let's start with a simplified version of the 
problem. If there is only 1 tiger (n =1), surely it will eat the sheep since it does not need 
to worry about being eaten. How about 2 tigers? Since both tigers are perfectly rational, 
either tiger probably would do some thinking as to what will happen if it eats the sheep. 
Either tiger is probably thinking: if I eat the sheep, I will become a sheep; and then I will 
be eaten by the other tiger. So to guarantee the highest likelihood of survival, neither 
tiger will eat the sheep. 


If there are 3 tigers, the sheep will be eaten since each tiger will realize that once it 
changes to a sheep, there will be 2 tigers left and it will not be eaten. So the first tiger 
that thinks this through will eat the sheep. If there are 4 tigers, each tiger will understand 
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that if it eats the sheep, it will turn to a sheep. Since there are 3 other tigers, it will be 
eaten. So to guarantee the highest likelihood of survival, no tiger will eat the sheep. 


Following the same logic, we can naturally show that if the number of tigers is even, the 
sheep will not be eaten. If the number is odd, the sheep will be eaten. For the case 
n=100, the sheep will not be eaten. 


2.2 Logic Reasoning 
River crossing 


Four people, A, B, C and D need to get across a river. The only way to cross the river is 
by an old bridge, which holds at most 2 people at a time. Being dark, they can't cross the 
bridge without a torch, of which they only have one. So each pair can only walk at the 
speed of the slower person. They need to get all of them across to the other side as 
quickly as possible. A is the slowest and takes 10 minutes to cross; B takes 5 minutes; C 
takes 2 minutes; and D takes | minute. 


What is the minimum time to get all of them across to the other side?’ 
Solution: The key point is to realize that the 10-minute person should go with the 5- 
minute person and this should not happen in the first crossing, otherwise one of them 


have to go back. So C and D should go across first (2 min); then send D back (1min); A 
and B go across (10 min); send C back (2min); C and D go across again (2 min). 


It takes 17 minutes in total. Alternatively, we can send C back first and then D back in 
the second round, which takes 17 minutes as well. 


Birthday problem 


You and your colleagues know that your boss A’s birthday is one of the following 10 
dates: 


Mar 4, Mar 5, Mar 8 
Jun 4, Jun 7 
Sep 1, Sep 5 
Dec 1, Dec 2, Dec 8 


A told you only the month of his birthday, and told your colleague C only the day. After 
that, you first said: “I don't know A’s birthday; C doesn't know it either." After hearing 


' Hint: The key is to realize that A and B should get across the bridge together. 
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what you said, C replied: “I didn’t know A’s birthday, but now I know it.” You smiled 
and said: “Now I know it, too.” After looking at the 10 dates and hearing your comments, 
your administrative assistant wrote down A’s birthday without asking any questions. So 
what did the assistant write? 


Solution: Don’t let the “he said, she said” part confuses you. Just interpret the logic 
behind each individual’s comments and try your best to derive useful information from 
these comments. 


Let D be the day of the month of A's birthday, we have De {1,2,4,5,7,8}. If the 


birthday is on a unique day, C will know the A’s birthday immediately. Among possible 
Ds, 2 and 7 are unique days. Considering that you are sure that C does not know A’s 
birthday, you must infer that the day the C was told of is not 2 or 7. Conclusion: the 
month is not June or December. (If the month had been June, the day C was told of may 
have been 2; if the month had been December, the day C was told of may have been 7.) 


Now C knows that the month must be either March or September. He immediately 
figures out A’s birthday, which means the day must be unique in the March and 
September list. It means A’s birthday cannot be Mar 5, or Sep 5. Conclusion: the 
birthday must be Mar 4, Mar 8 or Sep 1. 


Among these three possibilities left, Mar 4 and Mar 8 have the same month. So if the 
month you have is March, you still cannot figure out A’s birthday. Since you can figure 
out A’s birthday, A’s birthday must be Sep 1. Hence, the assistant must have written Sep 
l. 


Card game 


A casino offers a card game using a normal deck of 52 cards. The rule is that you turn 
over two cards each time. For each pair, if both are black, they go to the dealer’s pile; if 
both are red, they go to your pile; if one black and one red, they are discarded. The 
process is repeated until you two go through all 52 cards. If you have more cards in your 
pile, you win $100; otherwise (including ties) you get nothing. The casino allows you to 
negotiate the price you want to pay for the game. How much would you be willing to 
pay to play this game?” 


Solution: This surely is an insidious casino. No matter how the cards are arranged, you 
and the dealer will always have the same number of cards in your piles. Why? Because 
each pair of discarded cards have one black card and one red card, so equal number of 


* Hint: Try to approach the problem using symmetry. Each discarded pair has one black and one red card. 
What does that tell you as to the number of black and red cards in the rest two piles? 
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red and black cards are discarded. As a result, the number of red cards left for you and 
the number of black cards left for the dealer are always the same. The dealer always 
wins! So we should not pay anything to play the game. 


Burning ropes 


You have two ropes, each of which takes 1 hour to burn. But either rope has different 
densities at different points, so there's no guarantee of consistency in the time it takes 
different sections within the rope to burn. How do you use these two ropes to measure 45 
minutes? 


Solution: This is a classic brain teaser question. For a rope that takes x minutes to burn, 
if you light both ends of the rope simultaneously, it takes x/2 minutes to burn. So we 
should light both ends of the first rope and light one end of the second rope. 30 minutes 
later, the first rope will get completely burned, while that second rope now becomes a 
30-min rope. At that moment, we can light the second rope at the other end (with the 
first end still burning), and when it is burned out, the total time is exactly 45 minutes. 


Defective ball 


You have 12 identical balls. One of the balls is heavier OR lighter than the rest (you 
don't know which). Using just a balance that can only show you which side of the tray IS 
heavier, how can you determine which ball is the defective one with 3 measurements? 


Solution: 'This weighing problem is another classic brain teaser and is still being asked 
by many interviewers. The total number of balls often ranges from 8 to more than 100. 
Here we use n=12 to show the fundamental approach. The key is to separate the 
original group (as well as any intermediate subgroups) into three sets instead of two. The 
reason is that the comparison of the first two groups always gives information about the 
third group. 


Considering that the solution is wordy to explain, I draw a tree diagram in Figure 2.1 to 
show the approach in detail. Label the balls 1 through 12 and separate them to three 
groups with 4 balls each. Weigh balls 1, 2, 3, 4 against balls 5, 6, 7, 8. Then we go on to 


explore two possible scenarios: two groups balance, as expressed using an “=” sign, or 1, 


? Hint: First do it for 9 identical balls and use only 2 measurements, knowing that one is heavier than the 
rest. 
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2, 3, 4 are lighter than 5, 6, 7, 8, as expressed using an “<” sign. There is no need to 
explain the scenario that 1, 2, 3, 4 are heavier than 5, 6, 7, 8. (Why?) 


If the two groups balance, this immediately tells us that the defective ball is in 9, 10, 11 
and 12, and it is either lighter (L) or heavier (77) than other balls. Then we take 9, 10 and 
11 from group 3 and compare balls 9, 10 with 8, 11. Here we have already figured out 
that 8 is a normal ball. If 9, 10 are lighter, it must mean either 9 or 10 is L or 11 is H. In 
which case, we just compare 9 with 10. If 9 is lighter, 9 is the defective one and it is L; if 
9 and 10 balance, then 11 must be defective and H; If 9 is heavier, 10 is the defective 
one and it is L. If 9, 10 and 8, 11 balance, 12 1s the defective one. If 9, 10 is heavier, than 
either 9 or 10 is H, or 11 is L. 


You can easily follow the tree in Figure 2.1 for further analysis and it is clear from the 
tree that all possible scenarios can be resolved in 3 measurements. 


1/2/3/4 L or 5/6/7/8 H 9/10/11/12 L or H 


1/2L or 6H 4L or 7/8H D-O 9/10L or 11H 12L or 12H 9/10H or LIL 


OO Of : Oo OY OW 


HX TN / \ WN X 


IL 6H 11H 10L 12H I2L 10H 1IL 9H 


: 
6 
3 


Figure 2.1 Tree diagram to identify the defective ball in 12 balls 


In general if you have the information as to whether the defective ball is heavier or 


^ Here is where the symmetry idea comes in. Nothing makes the 1, 2, 3, 4 or 5, 6, 7, 8 labels special. If 1, 2, 
3, 4 are heavier than 5, 6, 7, 8, let's just exchange the labels of these two groups. Again we have the case 
of 1, 2, 3, 4 being lighter than 5, 6, 7, 8. 
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lighter, you can identify the defective ball among up to 3” balls using no more than n 
measurements since each weighing reduces the problem size by 2/3. If you have no 
information as to whether the defective ball is heavier or lighter, you can identify the 


defective ball among up to (3" —3)/2 balls using no more than n measurements. 


Trailing zeros 


How many trailing zeros are there in 100! (factorial of 100)? 


Solution: This is an easy problem. We know that each pair of 2 and 5 will give a trailing 
zero. If we perform prime number decomposition on all the numbers in 100!, it is 
obvious that the frequency of 2 will far outnumber of the frequency of 5. So the 
frequency of 5 determines the number of trailing zeros. Among numbers 1,2,---,99, and 


100, 20 numbers are divisible by 5 (5,10, ---, 100). Among these 20 numbers, 4 are 
divisible by 5^ (25, 50, 75,100). So the total frequency of 5 is 24 and there are 24 
trailing zeros. 


Horse race 


There are 25 horses, each of which runs at a constant speed that is different from the 
other horses'. Since the track only has 5 lanes, each race can have at most 5 horses. If 
you need to find the 3 fastest horses, what is the minimum number of races needed to 
identify them? 


Solution: This problem tests your basic analytical skills. To find the 3 fastest horses, 
surely all horses need to be tested. So a natural first step is to divide the horses to 5 
groups (with horses 1-5, 6-10, 11-15, 16-20, 21-25 in each group). After 5 races, we will 
have the order within each group, let's assume the order follows the order of numbers 
(e.g., 6 is the fastest and 10 is the slowest in the 6-10 group)" . That means 1, 6, 11, 16 
and 21 are the fastest within each group. 


Surely the last two horses within each group are eliminated. What else can we infer? We 
know that within each group, if the fastest horse ranks 5th or 4th among 25 horses, then 
all horses in that group cannot be in top 3; if it ranks the 3rd, no other horse in that group 
can be in the top 3; if it ranks the 2nd, then one other horse in that group may be in top 3; 
if it ranks the first, then two other horses in that group may be in top 3. 


? Such an assumption does not affect the generality of the solution. If the order is not as described, just 
change the labels of the horses. 
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So let’s race horses 1, 6, 11, 16 and 21. Again without loss of generality, let’s assume 
the order is 1, 6, 11, 16 and 21. Then we immediately know that horses 4-5, 8-10, 12-15, 
16-20 and 21-25 are eliminated. Since 1 is fastest among all the horses, 1 is in. We need 
to determine which two among horses 2, 3, 6, 7 and 11 are in top 3, which only takes one 
extra race. 


So all together we need 7 races (in 3 rounds) to identify the 3 fastest horses. 


Infinite sequence 


If x^x^x^x^x--- 22, where x^y 2 x, whatis x? 


Solution: This problem appears to be difficult, but a simple analysis will give an elegant 
solution. What do we have from the original equation? 


linx^x^x^x^x..-2o0lmx^x^x^x^x----2. In other words, as n œ, 
Ine n terms noo n—l terms 


adding or minus one x^ should yield the same result. 


SO X ox ^x^ wedex ou a puso 22 —x-42. 


2.3 Thinking Out of the Box 
Box packing 


Can you pack 53 bricks of dimensions 1x1x4 into a 6x6x6 box? 


Solution: This is a nice problem extended from a popular chess board problem. In that 
problem, you have a 8x8 chess board with two small squares at the opposite diagonal 
corners removed. You have many bricks with dimension 1x2. Can you pack 31 bricks 
into the remaining 62 squares? (An alternative question is whether you can cover all 62 
squares using bricks without any bricks overlapping with each other or sticking out of 
the board, which requires a similar analysis.) 


A real chess board figure surely helps the visualization. As shown in Figure 2.2, when a 
chess board is filled with alternative black and white squares, both squares at the 
opposite diagonal corners have the same color. If you put a 1x2 brick on the board, it 
will always cover one black square and one white square. Let's say it's the two black 
corner squares were removed, then the rest of the board can fit at most 30 bricks since 
we only have 30 black squares left (and each brick requires one black square). So to 
pack 31 bricks is out of the question. To cover all 62 squares without overlapping or 
overreaching, we must have exactly 31 bricks. Yet we have proved that 31 bricks cannot 
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fit in the 62 squares left, so you cannot find a way to fill in all 62 squares without 
overlapping or overreaching. 


Removed 


Removed <— 


Figure 2.2 Chess board with alternative black and white squares 


Just as any good trading strategy, if more and more people get to know it and replicate it, 
the effectiveness of such a strategy will disappear. As the chess board problem becomes 
popular, many interviewees simply commit it to memory (after all, it’s easy to remember 
the answer). So some ingenious interviewer came up with the newer version to test your 
thinking process, or at least your ability to extend your knowledge to new problems. 


If we look at the total volume in this 3D problem, 53 bricks have a volume of 212, which 
is smaller then the box’s volume 216. Yet we can show it is impossible to pack all the 
bricks into the box using a similar approach as the chess board problem. Let’s imagine 
that the 6x6x6 box is actually comprised of small 2x2x2 cubes. There should be 27 
small cubes. Similar to the chess board (but in 3D), imagine that we have black cubes 
and white cubes alternates—it does take a little 3D visualization. So we have either 14 
black cubes & 13 white cubes or 13 black cubes & 14 white cubes. For any 1x1x4 brick 
that we pack into the box, half (1x1x2) of it must be in a black 2x2x2 cube and the 
other half must be in a white 2x2x2 cube. The problem is that each 2x 2x2 cube can 
only be used by 4 of the 1x1x 4 bricks. So for the color with 13 cubes, be it black or 
white, we can only use them for 52 1x1x4 tubes. There is no way to place the 53th 
brick. So we cannot pack 53 bricks of dimensions 1x1x4 into a 6x6x6 box. 


Calendar cubes 


You just had two dice custom-made. Instead of numbers 1 — 6, you place single-digit 
numbers on the faces of each dice so that every morning you can arrange the dice in a 
way as to make the two front faces show the current day of the month. You must use 
both dice (in other words, days 1 — 9 must be shown as 01 — 09), but you can switch the 
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order of the dice if you want. What numbers do you have to put on the six faces of each 
of the two dice to achieve that? 


Solution: The days of a month include 11 and 22, so both dice must have 1 and 2. To 
express single-digit days, we need to have at least a 0 in one dice. Let's put a 0 in dice 
one first. Considering that we need to express all single digit days and dice two cannot 
have all the digits from 1 — 9, it’s necessary to have a 0 in dice two as well in order to 
express all single-digit days. 


So far we have assigned the following numbers: 


Dice one 


If we can assign all the rest of digits 3, 4, 5, 6, 7, 8, and 9 to the rest of the faces, the 
problem is solved. But there are 7 digits left. What can we do? Here’s where you need to 
think out of the box. We can use a 6 as a 9 since they will never be needed at the same 
time! So, simply put 3, 4, and 5 on one dice and 6, 7, and 8 on the other dice, and the 
final numbers on the two dice are: 


Dice one 


Door to offer 


You are facing two doors. One leads to your job offer and the other leads to exit. In front 
of either door is a guard. One guard always tells lies and the other always tells the truth. 
You can only ask one guard one yes/no question. Assuming you do want to get the job 
offer, what question will you ask? 


Solution: This is another classic brain teaser (maybe a little out-of-date in my opinion). 
One popular answer is to ask one guard: “Would the other guard say that you are 
guarding the door to the offer?” If he answers yes, choose the other door; if he answers 
no, choose the door this guard is standing in front of. 


There are two possible scenarios: 
1. Truth teller guards the door to offer; Liar guards the door to exit. 
2. Truth teller guards the door to exit; Liar guards the door to offer. 


If we ask a guard a direct question such as “Are you guarding the door to the offer?” For 
scenario 1, both guards will answer yes; for scenario 2, both guards will answer no. So a 
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direct question does not help us solve the problem. The key is to involve both guards in 
the questions as the popular answer does. For scenario 1, if we happen to choose the 
truth teller, he will answer no since the liar will say no; if we happen to choose the liar 
guard, he will answer yes since the truth teller will say no. For scenario 2, if we happen 
to choose the truth teller, he will answer yes since the liar will say yes; if we happen to 
choose the liar guard, he will answer no since the truth teller with say yes. So for both 
scenarios, if the answer is no, we choose that door; if the answer is yes, we choose the 
other door. 


Message delivery 


You need to communicate with your colleague in Greenwich via a messenger service. 
Your documents are sent in a padlock box. Unfortunately the messenger service is not 
secure, so anything inside an unlocked box will be lost (including any locks you place 
inside the box) during the delivery. The high-security padlocks you and your colleague 
each use have only one key which the person placing the lock owns. How can you 
securely send a document to your colleague?° 


Solution: If you have a document to deliver, clearly you cannot deliver it in an unlocked 
box. So the first step is to deliver it to Greenwich in a locked box. Since you are the 
person who has the key to that lock, your colleague cannot open the box to get the 
document. Somehow you need to remove the lock before he can get the document, 
which means the box should be sent back to you before your colleague can get the 
document. 


So what can he do before he sends back the box? He can place a second lock on the box, 
which he has the key to! Once the box is back to you, you remove your own lock and 
send the box back to your colleague. He opens his own lock and gets the document. 


Last ball 


A bag has 20 blue balls and 14 red balls. Each time you randomly take two balls out. 
(Assume each ball in the bag has equal probability of being taken). You do not put these 
two balls back. Instead, if both balls have the same color, you add a blue ball to the bag; 
if they have different colors, you add a red ball to the bag. Assume that you have an 
unlimited supply of blue and red balls, if you keep on repeating this process, what will 
be the color of the last ball left in the bag?’ What if the bag has 20 blue balls and 13 red 
balls instead? 


? Hint: You can have more than one lock on the box. 
7 Hint: Consider the changes in the number of red and blue balls after each step. 
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Solution: Once you understand the hint, this problem should be an easy one. Let (B, R) 


represent the number of blue balls and red balls in the bag. We can take a look what will 
happen after two balls are taken out. 


Both balls are blue: (B, R) > (B —1, R) 
Both balls are red: (B, R) > (B +1, R -2) 
One red and one blue: (B, R) > (B -1, R) 


Notice that R either stays the same or decreases by 2, so the number of red balls will 
never become odd if we begin with 14 red balls. We also know that the total number of 
balls decreases by one each time until only one ball is left. Combining the information 
we have, the last ball must be a blue one. Similarly, when we start with odd number of 
red balls, the final ball must be a red one. 


Light switches 


There is a light bulb inside a room and four switches outside. All switches are currently 
at off state and only one switch controls the light bulb. You may turn any number of 
switches on or off any number of times you want. How many times do you need to go 
into the room to figure out which switch controls the light bulb? 


Solution: You may have seen the classical version of this problem with 3 light bulbs 
inside the room and 3 switches outside. Although this problem is slightly modified, the 
approach is exact the same. Whether the light is on and off is binary, which only allows 
us to distinguish two switches. If we have another binary factor, there are 2x2=4 
possible combinations of scenarios, so we can distinguish 4 switches. Besides light, a 
light bulb also emits heat and becomes hot after the bulb has been lit for some time. So 
we can use the on/off and cold/hot combination to decide which one of the four switches 
controls the light. 


Turn on switches 1 and 2; move on to solve some other puzzles or do whatever you like 
for a while; turn off switch 2 and turn on switch 3; get into the room quickly, touch the 
bulb and observe whether the light is on or off. 


The light bulb is on and hot — switch 1 controls the light; 
The light bulb is off and hot — switch 2 controls the light; 
The light bulb is on and cold — switch 3 controls the light; 
The light bulb is off and cold — switch 4 controls the light. 
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Quant salary 


Eight quants from different banks are getting together for drinks. They are all interested 
in knowing the average salary of the group. Nevertheless, being cautious and humble 
individuals, everyone prefers not to disclose his or her own salary to the group. Can you 
come up with a strategy for the quants to calculate the average salary without knowing 
other people’s salaries? 


Solution: This is a light-hearted problem and has more than one answer. One approach is 
for the first quant to choose a random number, adds it to his/her salary and gives it to the 
second quant. The second quant will add his/her own salary to the result and give it to 
the third quant; ...; the eighth quant will add his/her own salary to the result and give it 
back to the first quant. Then the first quant will deduct the “random” number from the 
total and divide the “real” total by 8 to yield the average salary. 


You may be wondering whether this strategy has any use except being a good brain 
teaser to test interviewees. It does have applications in practice. For example, a third 
party data provider collect fund holding position data (securities owned by a fund and 
the number of shares) from all participating firms and then distribute the information 
back to participants. Surely most participants do not want others to figure out what they 
are holding. If each position in the fund has the same fund ID every day, it’s easy to 
reverse-engineer the fund from the holdings and to replicate the strategy. So different 
random numbers (or more exactly pseudo-random numbers since the provider knows 
what number is added to the fund ID of each position and complicated algorithm is 
involved to make the mapping one to one) are added to the fund ID of each position in 
the funds before distribution. As a result, the positions in the same fund appear to have 
different fund IDs. That prevents participants from re-constructing other funds. Using 
this approach, the participants can share market information and remain anonymous at 
the same time. 


2.4 Application of Symmetry 
Coin piles 


Suppose that you are blind-folded in a room and are told that there are 1000 coins on the 
floor. 980 of the coins have tails up and the other 20 coins have heads up. Can you 
separate the coins into two piles so to guarantee both piles have equal number of heads? 
Assume that you cannot tell a coin’s side by touching it, but you are allowed to turn over 
any number of coins. 


Solution: Let's say that we separate the 1000 coins into two piles with n coins in one pile 
and 1000 — 7 coins in the other. If there are m coins in the first pile with heads up, there 
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must be 20—m coins in the second pile with heads up. We also know that there are 
n —m coins in the first pile with tails up. We clearly cannot guarantee that m=10 by 
simply adjusting 7. 


What other options do we have? We can turn over coins 1f we want to. Since we have no 
way of knowing what a coin's side is, it won't guarantee anything if we selectively flip 
coins. However, if we flip all the coins in the first pile, all heads become tails and all 
tails become heads. As a result, it will have n-m heads and m tails (symmetry). So, to 
start, we need to make the number of tails in the original first pile equal to the number of 
heads in the second pile; in other words, to make n-m 220— m. n -20 makes the 
equation hold. If we take 20 coins at random and turn them all over, the number of heads 
among these turned-over 20 coins should be the same as the number of heads among the 
other 980 coins. 


Mislabeled bags 


Y ou are given three bags of fruits. One has apples in it; one has oranges in it; and one 
has a mix of apples and oranges in it. Each bag has a label on it (apple, orange or mix). 
Unfortunately, your manager tells you that ALL bags are mislabeled. Develop a strategy 
to identify the bags by taking out minimum number of fruits? You can take any number 
of fruits from any bags.* 


Solution: The key here is to use the fact that ALL bags are mislabeled. For example, a 
bag labeled with apple must contain either oranges only or a mix of oranges and apples. 
Let's look at the labels: orange, apple, mix (orange + apple). Have you realized that the 
orange label and the apple label are symmetric? If not, let me explain it in detail: If you 
pick a fruit from the bag with the orange label and it's an apple (orange — apple), then 
the bag is either all apples or a mix. If you pick a fruit from the bag with the apple label 
and it's an orange (apple — orange), then the bag is either an orange bag or a mix. 
Symmetric labels are not exciting and are unlikely to be the correct approach. So let's try 
the bag with the mix label and get one fruit from it. If the fruit we get 1s an orange, then 
we know that bag is actually orange (It cannot be a mix of oranges and apples since we 
know the bag's label is wrong). Since the bag with the apple label cannot be apple only, 
it must be the mix bag. And the bag with the orange label must be the apple bag. 
Similarly, for the case that apples are in the bag with the mix label, we can figure out all 
the bags using one single pick. 


* The problem struck me as a word game when I first saw it. But it does test a candidate's attention to 
details besides his or her logic reasoning skills. 
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Wise men 


A sultan has captured 50 wise men. He has a glass currently standing bottom down. 
Every minute he calls one of the wise men who can choose either to turn it over (set it 
upside down or bottom down) or to do nothing. The wise men will be called randomly, 
possibly for an infinite number of times. When someone called to the sultan correctly 
states that all wise men have already been called to the sultan at least once, everyone 
goes free. But if his statement is wrong, the sultan puts everyone to death. The wise men 
are allowed to communicate only once before they get imprisoned into separate rooms 
(one per room). Design a strategy that lets the wise men go free. 


Solution: For the strategy to work, one wise man, let’s call him the spokesman, will state 
that every one has been called. What does that tell us? 1. All the other 49 wise men are 
equivalent (symmetric). 2. The spokesman is different from the other 49 men. So 
naturally those 49 equivalent wise men should act in the same way and the spokesman 
should act differently. 


Here is one of such strategies: Every one of the 49 (equivalent) wise men should flip the 
glass upside down the first time that he sees the glass bottom down. He does nothing if 
the glass is already upside down or he has flipped the glass once. The spokesman should 
flip the glass bottom down each time he sees the glass upside down and he should do 
nothing if the glass is already bottom down. After he does the 49th flip, which means all 
the other 49 wise men have been called, he can declare that all the wise men have been 
called. 


2.5 Series Summation 


Here is a famous story about the legendary mathematician/physicist Gauss: When he 
was a child, his teacher gave the children a boring assignment to add the numbers from 1 
to 100. To the amazement of the teacher, Gauss turned in his answer in less than a 
minute. Here is his approach: 


X n=1 + 2+ +--+ 99+ 100 
+ + + u 
n=100+ 994+---+ 2 + 1 


E S v v d » 


29 n 210141011012 1012 101x100 > Yn = 


n=] n=) 


100x101 
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N 
This approach can be generalized to any integer N: > n= AOI ud) 
n=! 


The summation formula for consecutive squares may not be as intuitive: 


pee es A N 
ES 6 3 2 6 


N 
But if we correctly guess that `n’ =aN?°+bN°+cN+d and apply the initial 


n=) 


conditions 

N=0=> 0=d 
N=1>1=at+b+c+d 
N=2>5=8a+4b+2c+d 
N =3 > 14=27a4+9b+4+3c+d 


we will have the solution that a = 1/3, b = 1/2, c = 1/6, d = 0. We can then easily show 
that the same equation applies to all N by induction. 


Clock pieces 


A clock (numbered 1 — 12 clockwise) fell off the wall and broke into three pieces. You 
find that the sums of the numbers on each piece are equal. What are the numbers on each 
piece? (No strange-shaped piece is allowed.) 


, "E 12x13 
Solution: Using the summation equation, yn = 


n=] 
piece must sum up to 26. Some interviewees mistakenly assume that the numbers on 
each piece have to be continuous because no strange-shaped piece is allowed. It’s easy to 
see that 5, 6, 7 and 8 add up to 26. Then the interviewees’ thinking gets stuck because 
they cannot find more consecutive numbers that add up to 26. 


= 78. So the numbers on each 


Such an assumption is not correct since 12 and 1 are continuous on a clock. Once that 
wrong assumption is removed, it becomes clear that 12-1213 and 112-13. So the 
second piece is 11, 12, 1 and 2; the third piece is 3, 4, 9 and 10. 


Missing integers 


Suppose we have 98 distinct integers from 1 to 100. What is a good way to find out the 
two missing integers (within [1, 100])? 
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Solution: Denote the missing integers as x and y, and the existing ones are Z,,---,2Z,.. 
Applying the summation equations, we have 


100 98 
»n- MÀ —Xty- LS p 

n=| i=] 

100 3 2 98 
Yn =x +y 253 >x +y = Ls I ea 


n=) i=] 


i 
i=] 


Using these two equations, we can easily solve x and y. If you implement this strategy 
using a computer program, it is apparent that the algorithm has a complexity of O(n) for 
two missing integers in | ton. 


Counterfeit coins | 


There are 10 bags with 100 identical coins in each bag. In all bags but one, each coin 
weighs 10 grams. However, all the coins in the counterfeit bag weigh either 9 or 11 
grams. Can you find the counterfeit bag in only one weighing, using a digital scale that 
tells the exact weight? ” 


Solution: Yes, we can identify the counterfeit bag using one measurement. Take 1 coin 

out of the first bag, 2 out of the second bag, 3 out the third bag, ---, and 10 coins out of 
10 

the tenth bag. All together, there are »n =55 coins. If there were no counterfeit coins, 
i=] 

they should weigh 550 grams. Let’s assume the i-th bag is the counterfeit bag, there will 

be i counterfeit coins, so the final weight will be 550i. Since i is distinct for each bag, 

we can identify the counterfeit coin bag as well as whether the counterfeit coins are 

lighter or heavier than the real coins using 550 +i. 


This is not the only answer: we can choose other numbers of coins from each bag as long 
as they are all different numbers. 


Glass balls 


You are holding two glass balls in a 100-story building. If a ball is thrown out of the 
window, it will not break if the floor number is less than X, and it will always break if 


? Hint: In order to find the counterfeit coin bag in one weighing, the number of coins from each bag must 
be different. If we use the same number of coins from two bags, symmetry will prevent you from 
distinguish these two bags if one is the counterfeit coin bag. 


19 


Brain Teasers 


the floor number is equal to or greater than X. You would like to determine X. What is 
the strategy that will minimize the number of drops for the worst case scenario? ' 


Solution: Suppose that we have a strategy with a maximum of N throws. For the first 
throw of ball one, we can try the N-th floor. If the ball breaks, we can start to try the 
second ball from the first floor and increase the floor number by one until the second 
ball breaks. At most, there are N —1 floors to test. So a maximum of N throws are 
enough to cover all possibilities. If the first ball thrown out of N-th floor does not break, 
we have N -1 throws left. This time we can only increase the floor number by N —1 for 
the first ball since the second ball can only cover N —2 floors if the first ball breaks. If 
the first ball thrown out of (2N-1)th floor does not break, we have N —2 throws left. So 
we can only increase the floor number by N — 2 for the first ball since the second ball 
can only cover N —3 floors if the first ball breaks... 


Using such logic, we can see that the number of floors that these two balls can cover 
with a maximum of N throws is N +(N—1)+---+1=N(N +1)/2. In order to cover 100 


stories, we need to have N(N +1)/2 2100. Taking the smallest integer, we have N =14. 


Basically, we start the first ball on the 14th floor, if the ball breaks, we can use the 
second ball to try floors 1,2,---,13 with a maximum throws of 14 (when the 13th or the 
]4th floor is X). If the first ball does not break, we will try the first ball on the 
144+(14-1)=27th floor. If it breaks, we can use the second ball to cover floors 


15, 16, ---, 26 with a total maximum throws of 14 as well... 


2.6 The Pigeon Hole Principle 


Here is the basic version of the Pigeon Hole Principle: if you have fewer pigeon holes 
than pigeons and you put every pigeon in a pigeon hole, then at least one pigeon hole has 
more than one pigeon. Basically it says that if you have n holes and more than n+1 
pigeons, at least 2 pigeons have to share one of the holes. The generalized version is that 
if you have n holes and at least mn «1 pigeons, at least m+1 pigeons have to share one 
of the holes. These simple and intuitive ideas are surprisingly useful in many problems. 
Here we will use some examples to show their applications. 


'? Hint: Assume we design a strategy with M maximum throws. If the first ball is thrown once, the second 


ball can cover N —1 floors; if the first ball is thrown twice, the second ball can cover N — 2 floors... 
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Matching socks 


Your drawer contains 2 red socks, 20 yellow socks and 31 blue socks. Being a busy and 
absent-minded MIT student, you just randomly grab a number of socks out of the draw 
and try to find a matching pair. Assume each sock has equal probability of being 
selected, what is the minimum number of socks you need to grab in order to guarantee a 
pair of socks of the same color? 


Solution: This question is just a variation of the even simpler version of two-color-socks 
problem, in which case you only need 3. When you have 3 colors (3 pigeon holes), by 
the Pigeon Hole Principle, you will need to have 3+1=4 socks (4 pigeons) to guarantee 
that at least two socks have the same color (2 pigeons share a hole). 


Handshakes 


You are invited to a welcome party with 25 fellow team members. Each of the fellow 
members shakes hands with you to welcome you. Since a number of people in the room 
haven’t met each other, there’s a lot of random handshaking among others as well. If you 
don’t know the total number of handshakes, can you say with certainty that there are at 
least two people present who shook hands with exactly the same number of people? 


Solution: There are 26 people at the party and each shakes hands with from 1—since 
everyone shakes hands with you—to 25 people. In other words, there are 26 pigeons and 
25 holes. As a result, at least two people must have shaken hands with exactly the same 
number of people. 


Have we met before? 


Show me that, if there are 6 people at a party, then either at least 3 people met each other 
before the party, or at least 3 people were strangers before the party. 


Solution: This question appears to be a complex one and interviewees often get puzzled 
by what the interviewer exactly wants. But once you start to analyze possible scenarios, 
the answer becomes obvious. 


Let’s say that you are the 6th person at the party. Then by generalized Pigeon Hole 
Principle (Do we even need that for such an intuitive conclusion?), among the remaining 
5 people, we conclude that either at least 3 people met you or at least 3 people did not 
meet you. Now let’s explore these two mutually exclusive and collectively exhaustive 
scenarios: 


Case 1: Suppose that at least 3 people have met you before. 
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If two people in this group met each other, you and the pair (3 people) met each other. If 
no pair among these people met each other, then these people (23 people) did not meet 
each other. In either sub-case, the conclusion holds. 


Case 2: Suppose at least 3 people have not met you before. 


If two people in this group did not meet each other, you and the pair (3 people) did not 
meet each other. If all pairs among these people knew each other, then these people (2 3 
people) met each other. Again, in either sub-case, the conclusion holds. 


Ants on a square 


There are 51 ants on a square with side length of 1. If you have a glass with a radius of 
1/7, can you put your glass at a position on the square to guarantee that the glass 
encompasses at least 3 ants?! 


Solution: To guarantee that the glass encompasses at least 3 ants, we can separate the 
square into 25 smaller areas. Applying the generalized Pigeon Hole Principle, we can 
show that at least one of the areas must have at least 3 ants. So we only need to make 
sure that the glass is large enough to cover any of the 25 smaller areas. Simply separate 
the area into 5x5 smaller squares with side length of 1/5 each will do since a circle with 
radius of 1/7 can cover a square’? with side length 1/5. 


Counterfeit coins Il 


There are 5 bags with 100 coins in each bag. A coin can weigh 9 grams, 10 grams or 11 
grams. Each bag contains coins of equal weight, but we do not know what type of coins 
a bag contains. Y ou have a digital scale (the kind that tells the exact weight). How many 
times do you need to use the scale to determine which type of coin each bag contains?" 


Solution: If the answer for 5 bags is not obvious, let's start with the simplest version of 
the problem—1 bag. We only need to take one coin to weigh it. Now we can move on to 
2 bags. How many coins do we need to take from bag 2 in order to determine the coin 
types of bag 1 and bag 2? Considering that there are three possible types for bag 1, we 
will need three coins from bag 2; two coins won't do. For notation simplicity, let's 
change the number/weight for three types to -1, 0 and 1 (by removing the mean 10). If 


!! Hint: Separate the square into 25 smaller areas; then at least one area has 3 ants in it. 

7 A circle with radius r can cover a square with side length up to J2 r and J2 ~ 1.414. 

'^ Hint: Start with a simpler problem. What if you have two bags of coins instead of 5, how many coins do 
you need from each bag to find the type of coins in either bag? What is the minimum difference in coin 
numbers? Then how about three bags? 
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we only use 2 coins from bag 2, the final sum for 1 coin from bag 1 and 2 coins from 
bag 2 ranges from -3 to 3 (7 pigeon holes). At the same time we have 9 (3x3) possible 
combinations for the weights of coins in bag 1 and bag 2 (9 pigeons). So at least two 
combinations will yield the same final sum (9>7, so at least two pigeons need to share 
one hole), and we can not distinguish them. If we use 3 coins from bag 2, then the sum 
ranges from -4 to 4, which is possible to cover all 9 combinations. The following table 
exactly shows that all possible combinations yield different sums: 


Sum | 1 coin, bag 1 


Cland C2 represent the weights of coins from bag I and 2 respectively. 


Then how about 3 bags? We are going to have 3° = 27 possible combinations. Surely an 
indicator ranging from —13 to 13 will cover it and we will need 9 coins from bag 3. The 
possible combinations are shown in the following table: 


C1, C2, and C3 represent the weights of coins from bag 1, 2, and 3 respectively. 


Following this logic, it is easy to see that we will need 27 coins from bag 4 and 81 coins 
from bag 5. So the answer is to take 1, 3, 9, 27 and 81 coins from bags 1, 2, 3, 4, and 5, 
respectively, to determine which type of coins each bag contains using a single weighing. 


2./ Modular Arithmetic 


The modulo operation—denoted as x%y or x mod y—finds the remainder of division of 


number x by another number y. For simpicility, we only consider the case where y is a 
positive integer. For example, 5%3 = 2. An intuitive property of modulo operation is 
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that if x,%y=x,%y, then (x, — x, )9oy — 0. From this property we can also show that 
x%y, (x+1)%y, --, and (x y -1)96y are all different numbers. 


Prisoner problem 


One hundred prisoners are given the chance to be set free tomorrow. They are all told 
that each will be given a red or blue hat to wear. Each prisoner can see everyone else's 
hat but not his own. The hat colors are assigned randomly and once the hats are placed 
on top of each prisoner's head they cannot communicate with one another in any form, or 
else they are immediately executed. The prisoners will be called out in random order and 
the prisoner called out will guess the color of his hat. Each prisoner declares the color of 
his hat so that everyone else can hear it. If a prisoner guesses correctly the color of his 
hat, he is set free immediately; otherwise he is executed. 


They are given the night to come up with a strategy among themselves to save as many 
prisoners as possible. What is the best strategy they can adopt and how many prisoners 
can they guarantee to save?! 


Solution: At least 99 prisoners can be saved. 


The key lies in the first prisoner who can see everyone else's hat. He declares his hat to 
be red if the number of red hats he sees is odd. Otherwise he declares his hat to be blue. 
He will have a 1/2 chance of having guessed correctly. Everyone else is able to deduce 
his own hat color combining the knowledge whether the number of red hats is odd 
among 99 prisoners (excluding the first) and the color of the other 98 prisoners 
(excluding the first and himself). For example, if the number of red hats is odd among 
the other 99 prisoners. A prisoner wearing a red hat will see even number of red hats in 
the other 98 prisoners (excluding the first and himself) and deduce that he is wearing a 
red hat. 


The two-color case is easy, isn't it? What if there are 3 possible hat colors: red, blue, and 
white? What is the best strategy they can adopt and how many prisoners can they 
guarantee to save? 


Solution: The answer is still that at least 99 prisoners will be saved. The difference is 
that the first prisoner now only has 1/3 chance of survival. Let's use the following 
scoring system: red=0, green=1, and blue=2. The first prisoner counts the total score for 


^ Hint: The first prisoner can see the number of red and blue hats of all other 99 prisoners. One color has 
odd number of counts and the other has even number of counts. 

? Hint: That a number is odd simply means x%2 = 1 . Here we have 3 colors, so you may want to consider 
x%3 instead. 
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the rest of 99 prisoners and calculates s%3. If the remainder is 0, he announces red; if 
the remainder is 1, green; 2, blue. He has 1/3 chance of living, but all the rest of the 
prisoners can determine his own score (color) from the remainder. Let’s consider a 
prisoner i among 99 prisoners (excluding the first prisoner). He can calculate the total 


score (x) of all other 98 prisoners. Since (x+0)%3, (x+1)%3, and (x+2)%3 are all 


different, so from the remainder that the first prisoner gives (for the 99 prisoners 
including i), he can determine his own score (color). For example, if prisoner i sees that 
there are 32 red, 29 green and 37 blue in those 98 prisoners (excluding the first and 
himself). The total score of those 98 prisoners is 103. If the first prisoner announces that 
the remainder is 2 (green), then prisoner i knows his own color is green (1) since 
only 104%3 =2 among 103, 104 and 105. 


Theoretically, a similar strategy can be extended to any number of colors. Surely that 
requires all prisoners to have exceptional memory and calculation capability. 


Division by 9 
Given an arbitrary integer, come up with a rule to decide whether it is divisible by 9 and 
prove it. 


Solution: Hopefully you still remember the rules from your high school math class. Add 
up all the digits of the integer. If the sum is divisible by 9, then the integer is divisible by 
9; otherwise the integer is not divisible by 9. But how do we prove it? 

Let's express the original integer as a=a,10" +a,_,10"'+---+a,10'+a,. Basically we 
state that if a, +a,,+---+a,+a, =9x (x is a integer), then the a is divisible by 9 as 
well. The proof is straightforward: 

For any a=a,10"+a,,10"'+---+a10'+a,, let b=a—(a,+a,,+-:-+a,+a,). We 
have b-a,(10" -1) «a, (10! 21) »---—-a (10 —1)2a-9x, which is divisible by 9 
since all (10% — 1), k =1,-:-,n are divisible by 9. Because both b and 9x are divisible by 9, 
a — b «9x must be divisible by 9 as well. 

(Similarly you can also show that a =(-1)"a, - (-1)"'a, , --- (-1)'a, ay =11x is the 
necessary and sufficient condition for a to be divisible by 11.) 


25 


Brain Teasers 


Chameleon colors 


A remote island has three types of chameleons with the following population: 13 red 
chameleons, 15 green chameleons and 17 blue chameleons. Each time two chameleons 
with different colors meet, they would change their color to the third color. For example, 
if a green chameleon meets a red chameleon, they both change their color to blue. Is it 
ever possible for all chameleons to become the same color? Why or why not?! 


Solution: It is not possible for all chameleons to become the same color. There are 
several approaches to proving this conclusion. Here we discuss two of them. 


Approach 1. Since the numbers 13, 15 and 17 are “large” numbers, we can simplify the 
problem to 0, 2 and 4 for three colors. (To see this, you need to realize that if 
combination (m+1,n+1,p+l) can be converted to the same color, combination 


(m,n, p) can be converted to the same color as well.) Can a combination (0,2,4) be 
converted to a combination (0,0,6)? The answer is NO, as shown in Figure 2.3: 


(02.4) = 7 0,2:3) 


n mm 1, 5) prm 


Figure 2.3 chameleon color combination transitions from (0, 2, 4) 


Actually combination (1,2,3) is equivalent to combination (0,1, 2), which can only be 
converted to another (0,1,2) but will never reach (0,0,3). 


Approach 2. A different, and more fundamental approach, is to realize that in order for 
all the chameleons to become the same color, at certain intermediate stage, two colors 
must have the same number. To see this, just imagine the stage before a final stage. It 
must has the combination (1,1, x). For chameleons of two different colors to have the 


same number, their module of 3 must be the same as well. We start with 15 =3x, 
13 Z3y 41, and 17 =3z+2 chameleon, when two chameleons of different colors meet, 
we will have three possible scenarios: 


'© Hint: consider the numbers in module of 3. 
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(3x * 2,3y,3z +1) = (3x',3y'+1,3z'+ 2), one y meets one z 
(3x,3y +1,3z +2) > 4 (3(x -) 4 2,3(y + 1),3z +1) = (3x',3y'+1,3z'+ 2), one x meets one z 
(3(x -1) -2,3y,3(z +1) +1) =(3x',3y'+1,3z'+ 2), onex meetsone y 


So the pattern is preserved and we will never get two colors to have the same module of 
3. In other words, we cannot make two colors have the same number. As a result, the 
chameleons cannot become the same color. Essentially, the relative change of any pair of 
colors after two chameleons meet is either 0 or 3. In order for all the chameleons to 
become one color, at least one pair's difference must be a multiple of 3. 


2.8 Math Induction 


Induction is one of the most powerful and commonly-used proof techniques in 
mathematics, especially discrete mathematics. Many problems that involve integers can 
be solved using induction. The general steps for proof by induction are the following: 


= State that the proof uses induction and define an appropriate predicate P(n). 
= Prove the base case P(1), or any other smallest number n for the predicate to be true. 


" Prove that P(n) implies P(n+1) for every integer n. Alternatively, in a strong 
induction argument, you prove that P(l) P(2), -:-, and P(n) together imply 
P(n+1). 

In most cases, the real difficulty lies not in the induction step, but to formulate the 
problem as an induction problem and come up with the appropriate predicate P(n). The 
simplified version of the problem can often help you identify P(n). 


Coin split problem 

You split 1000 coins into two piles and count the number of coins in each pile. If there 
are x coins in pile one and y coins in pile two, you multiple x by y to get xy. Then you 
split both piles further, repeat the same counting and multiplication process, and add the 
new multiplication results to the original. For example, you split x to x, andx,, y to y, 
and y,, then the sum is xy + xx, + y,y,. The same process is repeated until you only 


have piles of 1 stone each. What is the final sum? (The final 1’s are not included in the 
sum.) Prove that you always get the same answer no matter how the piles are divided. 


27 


Brain Teasers 


Solution: Let n be the number of the coins and f(n) be the final sum. It is unlikely that 


a solution will jump to our mind since the number n 21000 is a large number. If you 
aren’t sure how to approach the problem, it never hurts to begin with the simplest cases 
and try to find a pattern. For this problem, the base case has n = 2. Clearly the only split 
is 1+1 and the final sum is 1. When n =3, the first split is 2+1 and we have xy =2 and 


the 2-coin pile will further give an extra multiplication result 1, so the final sum is 3. 
This analysis also gives the hint that when n coins are split into x and n —x coins, the 
total sum will be f(n) 2 x(n— x) f(x)+ f(n—x). 4 coins can be split into 2+2 or 
3+1. For either case we can apply x(n — x) *- f (x) - f (n — x) and yields the same final 
sum 6. 


TP ; : . n(n-] 
Claim: For 7 coins, independent of intermediate splits, the final sum is unes E 


So how do we prove it? The answer should be clear to you: by strong induction. We 
have proved the claim for the base cases n=2,3,4. Assume the claim is true for 


n=2,---,N-—1 coins, we need to prove that it holds for n = N coins as well. Again we 
apply the equation f (n) = x(n — x) *- f(x) * f (n— x). If N coins are split into x coins and 
N — x coins, we have 
fJ(N) 2 x(N - x) - f(x) + f(N - x) 
N(N -1) » (N-x(N-x-l N(N-1) 

2 2 
n(n —1) 

2 


= x(N -x)+ 


So indeed it holds for n = N as well and f(n)= is true for any n2 2. Applying 


the conclusion to n 21000, we have f (n) -1000x999/2. 


Chocolate bar problem 


A chocolate bar has 6 rows and 8 columns (48 small 1x1 squares). You break it into 
individual squares by making a number of breaks. Each time, break one rectangle into 
two smaller rectangles. For example, in the first step you can break the 6x8 chocolate 
bar into a 6x3 one and a 6x5 one. What is the total number of breaks needed in order 
to break the chocolate bar into 48 small squares? 


7 f(2) 21, f(3)- f(2) =2 and f(4) - f(3) =3 should give you enough hint to realize the pattern is 
n(n —1) 
VA ceto a 
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Solution: Let m be the number of the rows of the chocolate bar and n be the number of 
columns. Since there is nothing special for the case m=6 and n=8, we should find a 


general solution for all m and n. Let's begin with the base case where m =1 and n=1. 
The number of breaks needed is clearly 0. For m »1 and n =1, the number of breaks is 


m-l; similarly for m -1 and n » 1, the number of breaks is n—1. So for any m and n, 


if we break the chocolate into m rows first, which takes m -1 breaks, and then break 
each row into n small pieces, which takes m(n—1) breaks, the total number of breaks is 


(m —1) 4 m(n —1) 2 mn—1. If we breaks it into n columns first and then break each 
column into m small pieces, the total number of breaks is also mn —1. But is the total 
number of breaks always mn-1 for other sequences of breaks? Of course it is. We can 
prove it using strong induction. 

We have shown the number of breaks is mn-1 for base cases m2=1,n=1 and 
m —1,n2]1. To prove it for a general mxn case, let's assume the statement is true for 
cases where rows « m, columns € n and rows € m, columns <n. If the first break is 
along a row and it is broken into two smaller pieces mxn and mx(n—n,), then the 


total number of breaks is 1+(mxn,-1)+(mx(n-n,)—1)=mn-1. Here we use the 
results for rows € m, columns « n. Similarly, if it is broken into two pieces m, xn and 
(m — m,)xn, the total number of breaks is 1+(m,xn-1)+((m-—m,)xn-1)=mn-1. So 


the total number of breaks is always mn —1 in order to break the chocolate bar into 
mxn small pieces. For the case m=6 and n = 8, the number of breaks is 47. 


Although induction is the standard approach used to solve this problem, there is actually 
a simpler solution if you've noticed an important fact: the number of pieces always 
increases by 1 with each break since it always breaks one piece into two. In the 
beginning, we have a single piece. In the end, we will have mn pieces. So the number of 
breaks must be mn - 1. 


Race track 


Suppose that you are on a one-way circular race track. There are N gas cans randomly 
placed on different locations of the track and the total sum of the gas in these cans is 
enough for your car to run exactly one circle. Assume that your car has no gas in the gas 
tank initially, but you can put your car at any location on the track and you can pick up 
the gas cans along the way to fill in your gas tank. Can you always choose a starting 
position on the track so that your car can complete the entire circle?! 


'* Hint: Start with N = 1, 2 and solve the problem using induction. 
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Solution: If you get stuck as to how to solve the problem, again start with the simplest 
cases (N =1, 2) and consider using an induction approach. Without loss of generality, 


let's assume that the circle has circumference of 1. For N =1, the problem is trivial. Just 
start at where the gas can is. For N = 2, The problem is still simple. Let's use a figure to 
visualize the approach. As shown in Figure 2.4A, the amount of gas in can 1 and can 2, 
expressed as the distance the car can travel, are x, and x, respectively, so x, +x, =1. 
The corresponding segments are y, and y,, so y,+y,=1. Since x *x,-1 and 
y, +y, =1, we must have x, 2 y, or x; 2 y, (x, < y, and x, < y, cannot both be true). If 
X, 2 y, , We can start at gas can 1, which has enough gas to reach gas can 2, and get more 


gas from gas can 2 to finish the whole circle. Otherwise, we will just start at gas can 2 
and pick up gas can 1 along the way to finish the whole circle. 


yi 


yi 
A B 


Figure 2.4 Gas can locations on the cycle and segments between gas cans 


The argument for N =2 also gives us the hint for the induction step. Now we want to 
Show that if the statement holds for N =n, then the same statement also holds for 
N=n+l. As shown in Figure 2.4B, we have x,+x,+::-+x,,,=1 and 
y +y, + ty47lfor N=n+1. So there must exist at least one i,1<i<n+1, that 
has x, 2 y,. That means whenever the car reaches x,, it can reach x,,, with more gas 


(For i=n+1, it goes to i=] instead). In other words, we can actually “combine” x, and 


x,,, to one gas can at the position of x, with an amount of gas x, + x, (and eliminate 


+] 


the gas cani+1). But such combination reduces the N =n+1 problem to N =n, for 
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which the statement holds. So the statement also holds for N=n+1. Hence we can 
always choose a starting position on the track to complete the entire circle for any N. 


There is also an alternative approach to this problem that provides a solution to the 
starting point. Let’s imagine that you have another car with enough gas to finish the 
circle. You put that car at the position of a randomly chosen gas can and drive the car for 
a full circle. Whenever you reach a gas can (including at the initial position), you 
measure the amount of gas in your gas tank before you add the gas from the can to your 
gas tank. After you finish the circle, read through your measurement records and find the 
lowest measurement. The gas can position corresponding to the lowest measurement 
should be your starting position if the car has no gas initially. (It may take some thinking 
to fully understand this argument. I’d recommend that you again draw a figure and give 
this argument some careful thoughts if you don’t find the reasoning obvious.) 


2.9 Proof by Contradiction 


In a proof by contradiction or indirect proof, you show that if a proposition were false, 
then some logical contradiction or absurdity would follow. Thus, the proposition must be 
true. 


Irrational number 


Can you prove that X2. is an irrational number? A rational number is a number that can 
be expressed as a ratio of two integers; otherwise it is irrational. 


Solution: This is a classical example of proof by contradiction. If V2 is not an irrational 
number, it can be expressed as a ratio of two integers m and n. If m and n have any 
common factor, we can remove it by dividing both m and n by the common factor. So in 
the end, we will have a pair of m and n that have no common factors. (It is called 


irreducible fraction.) Since m/n 2 42, we have m? -2n^. So m? must be an even 
number and m must be an even number as well. Let's express m as 2x, where x is an 
integer, since m is even. Then m^ - 4x^ and we also have m^ -2x/, which means n 
must be even as well. But that both m and n are even contradicts the earlier statement 
that m and n have no common factors. So 4/2 must be an irrational number. 


Rainbow hats 


Seven prisoners are given the chance to be set free tomorrow. An executioner will put a 
hat on each prisoner's head. Each hat can be one of the seven colors of the rainbow and 
the hat colors are assigned completely at the executioner's discretion. Every prisoner can 
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see the hat colors of the other six prisoners, but not his own. They cannot communicate 
with others in any form, or else they are immediately executed. Then each prisoner 
writes down his guess of his own hat color. If at least one prisoner correctly guesses the 
color of his hat, they all will be set free immediately; otherwise they will be executed. 


They are given the night to come up with a strategy. Is there a strategy that they can 
guarantee that they will be set free?” 


Solution: This problem is often perceived to be more difficult than the prisoner problem 
in the modular arithmetic section. In the previous prisoner problem, the prisoners can 
hear others’ guesses. So one prisoner’s declaration gives all the necessary information 
other prisoners need. In this problem, prisoners won’t know what others’ guesses are. To 
solve the problem, it does require an aha moment. The key to the aha moment is given 


7 
by the hint. Once you realize that if we code the colors to 0-6, (Ys pe must be 


i=] 
among 0, 1, 2, 3, 4, 5 or 6 as well. Then each prisoner i—let’s label them as 0-6 as 
well—should give a guess g, so that the sum of g, and the rest of 6 prisoners’ hat color 


codes will give a remainder of i when divided by 7, where g, is a unique number 


kz0 


between 0 and 6. For example, prisoner 0’s guess should make C Da J = 0. 


This way, we can guarantee at least one of g, = x, for i=0,1,2,3,4,5,6. 


: 
We can easily prove this conclusion by contradiction. If g, + x,, then EIL zi 


izl 


(since g,+ p e i and g,and x, are both between 0 and 6). But if g, + x, for all 


kzi 


7 
i = 0,1,2,3,4,5, and 6, then PL # 0,1,2,3,4,5,6, which is clearly impossible. So 


i=] 
at least one of g, must equal to x,. As a result, using this strategy, they are guaranteed 
to be set free. 


Hint: Let’s assign the 7 colors of rainbow with code 0-6 and X, be the color code of prisoner i. Then 


(X 3 must be 0, 1, 2, 3, 4, 5 or 6. How many guesses can 7 prisoners make? 
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Chapter 3 Calculus and Linear Algebra 


Calculus and linear algebra lay the foundation for many advanced math topics used in 
quantitative finance. So be prepared to answer some calculus or linear algebra 
problems—many of them may be incorporated into more complex problems—in 
quantitative interviews. Since most of the tested calculus and linear algebra knowledge 
is easy to grasp, the marginal benefit far outweighs the time you spend brushing up your 
knowledge on key subjects. If your memory of calculus or linear algebra is a little rusty, 
spend some time reviewing your college textbooks! 


Needless to say, it is extremely difficult to condense any calculus/linear algebra books 
into one chapter. Neither is it my intention to do so. This chapter focuses only on some 
of the core concepts of calculus/linear algebra that are frequently occurring in 
quantitative interviews. And unless necessary, it does so without covering the proof, 
details or even caveats of these concepts. If you are not familiar with any of the concepts, 
please refer to your favorite calculus/linear algebra books for details. 


3.1 Limits and Derivatives 
Basics of derivatives 


Let’s begin with some basic definitions and equations used in limits and derivatives. 
Although the notations may be different, you can find these materials in any calculus 
textbook. 


Derivative: Let y= f(x), then f (x) =—= lim — Ay _ T (x * =e f(x) 


Ax>0 Ax = lim 


The product rule: If u =u(x) and v=v(x) and their respective derivatives exist, 
d(uv) Lu a du 


+v—, (uv)'=u'v+uv' 
ax dx dx 
The quotient rule: a(z) = pŽ —u Z) jy l Bi e i dinis 
dx v dx dx T v 
dy | dy du 
The chain rule: If and then —— 
y = f (u(x)) u —u(x), pug 


The generalized power rule: o = ny" T for Vn z0 


Some useful equations: 
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a =e" In(ab)=lna+lnb  e'-lim(1- 2)" 

lim + =| lim(1+ x)" =1+kx for any k 

lim(Inx/x")=0 for any r>0 limx’e* -0 foranyr 

d , „du da" " du d l du u' 
— e" =e" — — = (a" Ina) — —]lnu2——2— 
dx dx dx dx dx udx u 


d . d d s 
—sin x = COs x , —cosx--sinx, —tanx=sec x 
dx dx dx 


What is the derivative of y =Inx'"*?’ 


Solution: This is a good problem to test your knowledge of basic derivative formulas— 
specifically, the chain rule and the product rule. 


Let u = ln y = In(In x") - InxxIn(Inx). Applying the chain rule and the product rule, 


we have 


du d(Iy) l dy d(lnx) 
dx 


d (In(l 
«nln x) + In xx ZENG) _ Inn) , In x 


dx dx y dx x xInx 
To derive od ail we again use the chain rule by setting v=Inx: 
d(In(inx)) _ d(inv) dv _ 11 _ i 

dx dv dx v x xlnx 
250. MOD). , cs 0 day ee wel (In(In x) +1). 


pdx x xInx dx x 


Maximum and minimum 


Derivative f (x) is essentially the slope of the tangent line to the curve y= f(x) and 
the instantaneous rate of change (velocity) of y with respect to x. At point x=c, if 


' Hint: To calculate the derivative of functions with the format y = f(x)’, it is common to take natural 
logs on both sides and then take the derivative, since d(In y)/ dx 21/ y x dy/ dx. 
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f(c)>0, f(x) is an increasing function at c; if f'(c)<0, f(x) is a decreasing 
function at c. 

Local maximum or minimum: suppose that f(x) is differentiable at c and is defined 
on an open interval containing c. If f(c) is either a local maximum value or a local 
minimum value of f(x), then f (c) 20. 


Second Derivative test: Suppose the secondary derivative of f(x)  /f'(x), is 
continuous near c. If f (c) 2 0 and f"(c)>0, then f(x) has a local minimum at c; if 
f (c) 0 and f"(c) «O0, then. f(x) has a local maximum at c. 


Without calculating the numerical results, can you tell me which number is larger, e^ or 


aE 92 


Solution: Let's take natural logs of e” and z^. On the left side we have z Ine, on the 


; l lne Inz 
right side we have eln z. If e" » z^, e" » z^ < axIne>exlnz & — > ——. 
e c 


Inx . ; ; : ; 
Is it true? That depends on whether f(x) =—— is an increasing or decreasing function 
x 


l/xxx-Inx 1-Inx 


from e to z. Taking the derivative of f(x), we have f'(x)- 


x^ Io 
which is less than 0 when x>e (Inx>1). In fact, f(x) has global maximum when 
x=e forall x>0. So ic. inr and e^ » z*. 
e T 


Alternative approach: If you are familiar with the Taylor's series, which we will discuss 


: l a 
in Section 3.4, you can apply Taylor's series to e* : e* = "x ] Thiele LM So 
n. i 2k 3! 


} 


oo 


e! »1«x, Vx »0.Let x2 z/e-l, then e" */e»z/e c» e" >r el >n. 


L'Hospital's rule 
Suppose that functions f(x) and g(x) are differentiable at x — a and that lim g (a) z 0. 


Further suppose that lim /(a) -0 and limg(a)=0 or that lim f(a) — +œ and 


^ Hint: Again consider taking natural logs on both sides; Ina>Inb=>a>b since Inx is a 
monotonously increasing function. 
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lim g(a) — +00, then lim—— I(x) = lim A 
x4 g(x) *>4 g(x) 
an indeterminate form to a determinate form. 


L’Hospital’s rule converts the limit from 


What is the limit of e* / x? as x — œ, and what is the limit of x? In x as x > 0°? 


x00 


Solution: lim = is a typical example of L’Hospital’s rule since lime" =o and 
o x^ 


lim x^ =œ. Applying L'Hospital's rule, we have 


The result still has the property that lim f(x) = lime" =œ and lim g(x) = lim 2x = oo, so 
we can apply the L’ Hospital's rule again: 


200. dum osdimc 09 oim C equ ete ua es 
x> g(x) x x xm g (x) x3 Dy | x d(2x)/ dx x 2 


= OO. 


At first look, L'Hospital's rule does not appear to be applicable to lim x^ In x since it’s 


= a . lnx 

not in the format of lim—— L oi x) . However, we can rewrite the original limit as lim —7- 
x2 g(x x30! x 

and it becomes obvious that lim x ^ =œ and lim In x » —. So we can now apply 


x—0* x-0* 


L'Hospital's rule: 


2 
lim x? ^"Inx- lim 22 agg eee ua T aedi t eg 
PME x^ x d(x*)ldx »«-2/x! xo*-2 
3.2 Integration 


Basics of integration 
Again, let's begin with some basic definitions and equations used in integration. 


If we can find a function F(x) with derivative f(x), then we call F(x) an 
antiderivative of f(x). 


If fQ) 2 FG). [ £0) = [ F'G9dx = [FOL = FD) Fla) 
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dF’ (x) _ 


= HL) F@=y, => F= y,* [ fd 


k+l 
The generalized power rule in reverse: Ju\du ="——+¢ (k #1), where c is any 
+ 


constant. 


Integration by substitution: 
[/ (G0): 'G)dx = |f(u)du with u= g(x), du =g (x)dx 


Substitution in definite integrals: f fle(x)) -2 '(x)dx = [. ^ f (u)du 


Integration by parts: [uav =uv— fvdu 


A. What is the integral of In(x)? 


Solution: This is an example of integration by parts. Let u =lnx and v=x, we have 
d(uv) = vdu + udv = (xx1/ x)dx + ln xax, 


5 fin xdx = xln x- fax =xlnx-x+c, where c is any constant. 


B. What is the integral of sec(x) from x = 0 tox = 27/6? 


Solution: Clearly this problem is directly related to differentiation/integration of 
trigonometric functions. Although there are derivative functions for all basic 


: : d . 
trigonometric functions, we only need to remember two of them: 2 D CPUS x; 


d l ) i i 
2:09 x — —sin x. The rest can be derived using the product rule or the quotient rule. For 


example, 

dsecx d(l/cosx) sinx 

—— AE ES SEO alor 
dx dx COS X 

dtanx d(sinx/cosx) cos x-«sin^x _ T 
dx dx cos? x 
d(sec x + tan x) 

l LS - = = sec x(sec x + tan x). 
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Since the (sec x + tan x) term occurs in the derivative, we also have 


din|secx+tanx|  secx(secx + tan x) = 
ax (sec x + tan x) 


ecx 
> [sec x =]n | sec x + tan x | +c 


and [ i sec x = In(sec(z / 6) 4 tan(z / 6)) —In(sec(0) + tan(0)) = In(/3 ) 


Applications of integration 


A. Suppose that two cylinders each with radius 1 intersect at right angles and their 
centers also intersect. What is the volume of the intersection? 


Solution: This problem is an application of integration to volume calculation. For these 
applied problems, the most difficult part is to correctly formulate the integration. The 


general integration function to calculate 3D volume is V = f A(z)dz where A(z) is the 


cross-sectional area of the solid cut by a plane perpendicular to the z-axis at coordinate z. 
The key here is to find the nght expression for cross-sectional area A as a function of z. 


Figure 3.1 gives us a clue. If you cut the intersection by a horizontal plane, the cut will 
be a square with side-length (2r) -(2zy. Taking advantage of symmetry, we can 


calculate the total volume as 


2x [ (2r -(22). ge -8x[rz- 2 /3], =16/37? = 16/3. 


An alternative approach requires even better 3D imagination. Let's imagine a sphere that 
is inscribed inside both cylinders, so it is inscribed inside the intersection as well. The 
sphere should have a radius of r/2. At each cut perpendicular to the z-axis, the circle 
from the sphere is inscribed in the square from the intersection as well. So 
A, =4A Since it’s true for all z values, we have 


circle ~ 4 ^" square* 


=> Vv. -16/3r? =16/3. 


int ersection 


ae x iz(5y xs ri 


V er section 
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Figure 3.1 Interaction of two cylinders 


B. The snow began to fall some time before noon at a constant rate. The city of 
Cambridge sent out a snow plow at noon to clear Massachusetts Avenue from MIT to 
Harvard. The plow removed snow at a constant volume per minute. At | pm, it had 
moved 2 miles and at 2 pm, 3 miles. When did the snow begin to fall? 


Solution: Let's denote noon as time 0 and assume snow began to fall 7 hours before 
noon. The speed at which the plow moves is inversely related to the vertical cross- 
sectional area of the snow: v — c, / A(t), where v is the speed of the plow, c, is a constant 
representing the volume of snow that the plow can remove every hour and A(/) is the 
cross-sectional area of the snow. If ¢ is defined as the time after noon, we also have 
A(t) 7 c, (t - T) , where c, is the rate of cross-sectional area increase per hour (since the 


$ where c =<". Taking the 


snow falls at a constant rate). So v - ———— - —— 
C(t+T) t+T C, 


integration, we have 


[z5-d- ema +7)-cin7 =cin{ ET. ]- 2 
T +t T 


[5 zai ei D)-ctnr - cn (277 ]- 3 
T «t T 
From these two equations, we get 


ler 2+T ° 2 | 
m (GE =T -T41-02 T =(V5-1)/2. 
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Overall, this question, although fairly straightforward, tests analytical skills, integration 
knowledge and algebra knowledge. 


Expected value using integration 


Integration is used extensively to calculate the unconditional or conditional expected 
value of continuous random variables. In Chapter 4, we will demonstrate its value in 
probability and statistics. Here we just use one example to show its application: 


If X is a standard normal random variable, X ~ N(0, 1), what is E[ X | X » 0]? 


Solution: Since X ~ N(0, 1), the probability density function of x is f(x) = he" 


and we have E[X | X » 0]= [ Fad = ESTI 


Because d(-1/2x^)- —x and [e'ay =e" +c, where c is an arbitrary constant, it is 


obvious that we can use integration by substitution by letting u — —1/2x^. Replace 
-2x? 


e with e" and xdx with —du, we have 
e 2x" Boll cd Dog cx edu u | 
[x xe dx = [ -e du le ji = 5; 0 ==, where le [i is 


determined by x=O>u=0 and x =œ => u = —o 


. E[X | X »0]2 1/ V27 


3.3 Partial Derivatives and Multiple Integrals 
J (Xo + Ax, yo) — F(X» Yo) 


Gaia fe Of m B 
Partial derivative: w= f(x, y) > pu Yo) = lim n - f. 
2 2 
Second order partial derivatives: A ) P REI —)-— o (S ) 


Ox! ax ax” Oxdy ax Oy by &x 
The general chain rule: Suppose that w= f oe and that each of variables 


X, X, , X, is a function of the variables ¢,, ¢,, ---, ¢,. If all these functions have 


Ow _ Ow ar Ow Ox, Ow Ox 
continuous first-order partial derivatives, then — = —— — + ——~++---+——— 
Ot, — 6x Ot, Ox, Ot, Ox,, Ot 


1 i 


each i, 1X i&n. 
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Changing Cartesian integrals into polar integrals: The variables in two-dimension 
plane can be mapped into polar coordinates: x =rcos@, y=rsin@. Tthe integration in a 


continuous polar region R is converted to 


[£65 y)dxdy = || f(r. cos 0,rsin 0) r dr dO. 


Calculate [ e "dx. 


Solution: Hopefully you happen to remember that the probability density function (pdf) 


of the standard normal distribution is f(x) = we ^. By definition, we have 
T 


[| Sod = [mee - [pe =1 => [ends - i 


If you've forgotten the pdf of the standard normal distribution or if you are specifically 


1 
asked to prove [ 
V27 


e" "dx -1, you will need to use polar integrals to solve the 


problem: 


pep e? "1gy x É easy 2 [ [ g (eos sn sin? 0)/2 1 19 


= [ [e rardo=- [' e" ^ac 12) [^ a6 
= -[e"^ | [er mM an 


Since [er "ax = [6 "dy, we have | e Pde = 2 > [e^ V. 


3.4 Important Calculus Methods 


Taylor's series 


One-dimensional Taylor's series expands function f(x) as the sum of a series using the 
derivatives at a point x — x, : 


E a a X J 608) 


F(x) = f (Xp) f G4) — 3) + (x ~ x) HE (XH) + 
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Oa, SOO a, 
n! 


If x, =0, f(x) = f0) + f (00x += 


Taylor’s series are often used to represent functions in power series terms. For example, 
Taylor's series for three common transcendental functions, e*, sinx and cosx, at 


X, =0 are 

-—— i x 

ec = — = — + — . 

2 I! 2! 3! 

n .,2n«4l 3 5 7 

sin x = cer ae Sys p c au. 

(2n 4 1)! 3! 5! 7! 

y x?" 2 4 6 

COS x = y= t pc 

"o (2n)! 21 4! 6! 


The Taylor's series can also be expressed as the sum of the nth-degree Taylor 


T 5 Ls) f 9 
i) 


polynomial T (x)= f(x,)+ f '(x,)(x — x4) += (x — x,)^ 4 ——^(x-x,)" and 


a remainder R (x): f(x)- T, (x) R,(x). 


fd 
(n 4 1)! 


| f eg) for all X between x, and x, we get constraint 


n+l 


For some x between x, and x, R, (x)= |x- xX | . Let M be the maximum of 


Mx|x-x, E 
Ce 
(n+1)! 


A. What is ï? 


Solution: The solution to this problem uses Euler’s formula, e" =cos@+isin@, which 
can be proven using Taylor’s series. Let’s look at the proof. Applying Taylor’s series to 
e^, cos@ and sin@, we have 


A . 2 : 3 " 4 2 3 4 : 
pug UO Ur QU js es m OQ © 0 m 


=1+{/——-— -i— +— +i—+ 
I! 2! 3! 4! I! 2! 3! 4! 5! 
2 4 6 
cos @ = NN NE) 
2! 4! 6! 
3 5 7 3 5 7 
SU TEL ANE LM M sung ad Lr NP AMI 
3! 5! 7! I! 31 5! 7! 
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Combining these three series, it is apparent that e° = cos@+isin 8. 


When @=2, the equation becomes e" -cosz *isinz 2-1. When @=7/2, the 


equation becomes e^" -cos(z7/2)«isin(z/2) 2 i? So Ini = In(e"? )- iz/2. 


Hence, In(i)- ilni-i(iz/2) 2 -2/22 i 2 e 7^? 


B. Prove (14 x)' 2 14 nx for all x » -1 and for all integers n 22. 


Solution: Let f(x) 2 (14 x)". It is clear that 1+ nx is the first two terms in the Taylor's 
series of f(x) with x, 20. So we can consider solving this problem using Taylor's 
series. 

For x, =0 we have (14 x)' «1 for Vn 2 2. The first and secondary derivatives of f(x) 
are f'(x) 2 n(14- x)! and f'(x) 2 n(n-D(1n4 x)"?^. Applying Taylor's series, we have 

f' E x) f' = x? 


9 


F(X) = fes) f 09) 7x) + (x-x)' = f(0)+ f Ox + 


=l+nx+n(n—-1)14+x)"? x? 
where x x €0 if x «0 and x 2 x 20 if x>0. 
Since x »-1 andn 22, we have n » 0, (n-1) » 0, (1+3)? » 0, x? 20. 


Hence, n(n-1)(1-- X) ^x 0 and f(x)2 (19 x)" >l+nx. 


If Taylor's series does not jump to your mind, the condition that n is an integer may give 
you the hint that you can try the induction method. We can rephrase the problem as: for 


every integer n 2 2 , prove (1+ x)” 214 nx for x» -1. 


The base case: show (14 x)' 2 1-2- nx, Vx » -1 when n -2, which can be easily proven 
since (1- x 21- 2x 4 x 214 2x, Vx »-1. 


The induction step: show that if (l+x)” 2l+mx,Vx>-1 when n=k, the same 


statement holds for n=k+1:  (1-x)"'21-(k4Dx,Vx»-1. This step is 
straightforward as well. 


? Clearly they satisfy equation (e"" ) =} =e" =-1, 
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(1+ x)! =(14+ x)‘ (1+) 
>(1+kx)(14+ x) =14(k4+)x+h’, Vx»-1 
21+(k+1)x 


So the statement holds for all integers n 2 2 when x» -1. 


Newton's method 


Newton's method, also known as the Newton-Raphson method or the Newton-Fourier 
method, is an iterative process for solving the equation f(x) =0. It begins with an initial 


n to solve f(x) x 0 if x,x»,,--- 


value x, and applies the iterative step x,,, =x 


converge." 


Convergence of Newton’s method is not guaranteed, especially when the starting point 
is far away from the correct solution. For Newton's method to converge, it is often 
necessary that the initial point is sufficiently close to the root; f(x) must be 


differentiable around the root. When it does converge, the convergence rate is quadratic, 


Xl = x, | 


which means ; 
x, —X,) 


<6 <1, where x, is the solution to f(x) =0. 


A. Solve x^ =37 to the third digit. 


Solution: Let f(x) 2 x! -37, the original problem is equivalent to solving f(x) =0. 
Xy = 6 is a natural initial guess. Applying Newton's method, we have 


ur E 
J(%) _ -2H = 6 - =" = 6.083. 


x 
ee iC? 25 
(6.0837 = 37.00289, which is very close to 37.) 


If you do not remember Newton’s method, you can directly apply Taylor’s series for 
function f(x)=Vx with f'(x) 2 4x^^: 


f (37) = £36) + f (36)(37 -36) 2 61/12 = 6.083. 


* The iteration equation comes from the first-order Taylor’s series: 


| f(x,) 
fo, s f(x)tf (x, (x,,,-*,)=9> x,, xL m 
f Xx,) 
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Alternatively, we can use algebra since it is obvious that the solution should be slightly 
higher than 6. We have (6+ y)! =37 > y! +12y-1=0. If we ignore the y^ term, 
which is small, then y 20.083 and x =6+ y = 6.083. 


B. Could you explain some root-finding algorithms to solve f(x) -0? Assume f(x) is 
a differentiable function. 


Solution: Besides Newton's method, the bisection method and the secant method are two 
alternative methods for root-finding.? 


Bisection method is an intuitive root-finding algorithm. It starts with two initial values 
a,and b, such that f(a,)<0 and f(b,) » 0. Since f(x) is differentiable, there must be 
an x between a, and b, that makes f(x)=0. At each step, we check the sign of 
f((a,*b,)/2). If f((a,*5,)/2)«0, we set b,, -b, and a,,-(a,* b,)/2; If 
f ((a, * b,)/ 2) » 0, we set a,,, =a, and b,,, — (a, +b,)/2; If f ((a, * b,)/2) =0, or its 
absolute value is within allowable error, the iteration stops and x= (a,  5,)/2. The 


+! 


; Xn X . oe 
bisection method converges linearly, “+ < 6 «1, which means it is slower than 


Ta ey 


Newton's method. But once you find an a,/b, pair, convergence is guaranteed. 


Secant method starts with two initial values x,, x, and applies the iterative step 


b =x, —-— n a f(x,). It replaces the f'(x,) in Newton's method with a 
SOn) f Q4) 


F(x) - f (5,4) 


X, —xX 


n n-| 


linear approximation . Compared with Newton’s method, it does not 


require the calculation of derivative f'(x,), which makes it valuable if f'(x) is difficult 
to calculate. Its convergence rate 1s (1+ V5 y 2, which makes it faster than the bisection 


method but slower than Newton's method. Similar to Newton's method, convergence is 
not guaranteed if initial values are not close to the root. 


Lagrange multipliers 


The method of Lagrange multipliers is a common technique used to find local 
maximums/minimums of a multivariate function with one or more constraints. 


> Newton’s method is also used in optimization—including multi-dimensional optimization problems—to 
find local minimums or maximums. 
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Let f(x, x,,°::, x,) be a function of n variables x=(x,, x,,--, x,) with gradient 
vector Vf(x)- (Z, Uy 21 ) The necessary condition for maximizing or 
minimizing f(x) subject to a set of k constraints 

g,(%,,%,,°°°,%,) = 0, £;(X,,X3,,:7,x,) = 9, niet g, (XXX) =9 


is that Vf (x) x AVg,(x)- A,Vg; (x) ----- A, Vg, (x) 30, where A,---,A4, are called the 
Lagrange multipliers. 


What is the distance from the origin to the plane 2x - 3y - 4z 2 12? 


Solution: The distance (D) from the origin to a plane is the minimum distance between 
the origin and points on the plane. Mathematically, the problem can be expressed as 


min D! = f(x,y,z) =x +y « z? 
st. g(x, y,Z)=2x+3y+4z-12=0 
Applying the Lagrange multipliers, we have 


Y AS —2x42A4-0 A=-24/29 


9 LAS —2y-3A-0 x - 24/29 2 7 ; 12 
a 71 236/29 2 P7 X9) +5) +5) = 
YAT -2x-44-0 y- 29 


2x+3y+4z-12=0 z=48/29 


In general, for a plane with equation ax+by+cz=d, the distance to the origin is 
|a| 


Na +b +e? 


pa 


3.5 Ordinary Differential Equations 


In this section, we cover four typical differential equation patterns that are commonly 
seen in interviews. 


* The method of Lagrange multipliers is a special case of Karush-Kuhn-Tucker (KKT) conditions, which 
reveals the necessary conditions for the solutions to constrained nonlinear optimization problems. 
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Separable differential equations 


A separable differential equation has the form 2 = g(x)h(y). Since it is separable, we 
can express the original equation as PES = g(x)dx. Integrating both sides, we have the 
y 
solution [ 2A = [e(a)ae. 
h(y) 


A. Solve ordinary differential equation y'+6xy =0, y(0) 21 


Solution: Let g(x)=-6x and h(y)= y, we have OY Fhe Integrate both sides of 
y 


-3x? +e 


the equation: p = [-6xdx => Iny=-3x°+c > y=e , where c is a constant. 
y 


Plugging in the initial condition y(0)=1, we have c=0 and y= gu 


B. Solve ordinary differential equation y'= E 
x 


+y 


Solution: Unlike the last example, this equation is not separable in its current form. But 
we can use a change of variable to turn it into a separable differential equation. Let 
z = x+ y, then the original differential equation is converted to 


A ag E EE TRE E HR [zdz = f2xdx+c 
dx Z dx Ž 


— (x4 yy =z =2x +c > y! +2xy-x =c 
First-order linear differential equations 


A first-order differential linear equation has the form 2, P(x)y = Q(x). The standard 


approach to solving a first-order differential equation is to identify a suitable function 
I(x), called an integrating factor, such that J(x)(y'+ P(x)y)=J(x)y'+1(x)P(x)y 


’ Hint: Introduce variable z = x + y. 
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- (1(x)y)'; Then we have (/(x)y)'=/(x)Q(x) and we can integrate both sides to solve 


I(x)Q(x)dx 
for y: I(x)y= [6909094 => y= Posos 
I(x) 
: : dl Za : ; 
The integrating factor, I(x), must satisfy ——~=J/(x)P(x), which means /(x) is a 
separable differential equation with general solution /(x) = pe : 
Solve ordinary different equation y'+ Z= =: , Y(D =1, where x > 0. 


Solution: This is a typical example of first-order linear equations with P(x) Et and 
x 


o Po - Ius 


Q(x) = —. So I(x) = 


=e" =x and we have I(x)Q(x) = = 
x 


m rm = (xy)'=1(x)Q(x) =1/x 


Taking integration on both sides, xy = {a /x)dx=|Inx+c> y= ee 
x 
In x+ 1 
Plugging in y(1)=1, we get c=1 and y = ——— 
x 


Homogeneous linear equations 


A a I equation is a second-order differential equation with the form 


It is easy to show that, if y, and y, are linearly independent solutions to the 
homogeneous linear equation, then any y(x)=c,y,(x)+c,y,(x), where c, and c, are 
arbitrary constants, is a solution to the homogeneous linear equation as well. 


When a, b and c (a 20) are constants instead of functions of x, the homogenous 
linear equation has closed form solutions: 


Let r, and r, be the roots of the characteristic equation ar^ + br+c=0, 


" The constant c is not needed in this case since it just scales both sides of the equation by a factor. 
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1. If r, and r, are real and r z r,, then the general solution is y 2c,e" +c,e'; 
2. If r, and r, are real and r =r, =r, then the general solution is y 2 ce" *c,xe" ; 


3. If n and r, are complex numbers @+tif, then the general solution is 
y =e% (c, cos Bx * c, sin Bx). 


It is easy to verify that the general solutions indeed satisfy the homogeneous linear 
solutions by taking the first and secondary derivatives of the general solutions. 


What is the solution of ordinary differential equation y"+ y'+ y 2 0? 


Solution: In this specific case, we have a=b = c =1 and b^ — 4ac = -3« 0, so we have 
complex roots r = —1/ 2+/3/2i (a= -1/2, B= 43/2 ), and the general solution to the 
differential equation 1s therefore 


y =e" (c, cos Bx +c, sin Bx) =e" (c cos( 3 / 2x) + C, sin(V3 / 2x)| , 


Nonhomogeneous linear equations 


2 
d 7 ph cy 2 0, a nonhomogeneous linear 
dx dx 


Unlike a homogenous linear equation a 


2 
equation a5 2 eb ey e d(x) has no closed-form solution. But if we can find a 


d? 
" i ` +cy =d(x), then y=y,(x)+ y, (x), where 


particular solution y,(x) for a 


2 
E J (529 + cy=0, isa 
ax ax 


y,(x) is the general solution of the homogeneous equation a 


2 
general solution of the nonhomogeneous equation a " +b n +cy=d(x). 


—b € X b! — 4ac 
2a 
should either commit the formula to memory or be able to derive it using (r - b/2ay - (b — 4ac)/4a'. 


? A quadratic equation ar^ + br «c - 0 has roots given by quadratic formula + = You 
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Although it may be difficult to identify a particular solution y,(x) in general, in the 


special case when d(x) is a simple polynomial, the particular solution is often a 
polynomial of the same degree. 


What is the solution of ODEs y"+ y'+ y zl and y"+ y'+y=x? 


Solution: In these ODEs, we again have a= b =c «1 and b? —4ac = -3« 0, so we have 
complex solutions r = —1/ 2+/3/2i (~a=-1/2, B= 434 2) and the general solution is 


y €" (c cos(3 / 2x) +c, sin(V3 / 2x). 


What is a particular solution for y"+ y'- y=1? Clearly y 21 is. So the solution to 
y"t+y'ty=l is 


y = y, (x) + y, (x) = €" (a cos(V3 / 2x) +, sin( V3 /23)) 1. 
To find a particular solution for y" y'+ y =x, Let y,(x)=mx+n, then we have 


y"+y'+y=0+m+(mx+n)=x= m=1,n=-1. So the particular solution is x—1 and 
the solution to y"+ y'+ y=x Is 


y=y,(x)+y,(x) =e" (o cos(43 / 2x) * c, sin(V3 / 2x) £ (x - 1). 


3.6 Linear Algebra 


Linear algebra is extensively used in applied quantitative finance because of its role in 
statistics, optimization, Monte Carlo simulation, signal processing, etc. Not surprisingly, 
it is also a comprehensive mathematical field that covers many topics. In this section, we 
discuss several topics that have significant applications in statistics and numerical 
methods. 


Vectors 


An nx] (column) vector is a one-dimensional array. It can represent the coordinates of 
a point in the R” (n-dimensional) Euclidean space. 
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Inner product/dot product: the inner product (or dot product) of two R" vectors x and 


y is defined as 5 x,y, =x’ y 


i=] 


Euclidean norm: M = Y ax xs lx -y| = V(x -y) (x-y) 
i=] 


T 
Then angle 0 between R” vectors x and y has the property that cos@ = >Y x and y 


ll 
are orthogonal if x’ y 2 0. The correlation coefficient of two random variables can be 
viewed as the cosine of the angle between them in Euclidean space ( p =cos@ ). 


There are 3 random variables x, y and z. The correlation between x and y is 0.8 and the 
correlation between x and z is 0.8. What is the maximum and minimum correlation 
between y and z? 


Solution: We can consider random variables x, y and z as vectors. Let 0 be the angle 
between x and y, then we have cosÓ = p, , — 0.8. Similarly the angle between x and z is 
O as well. For y and z to have the maximum correlation, the angle between them needs 
to be the smallest. In this case, the minimum angle is 0 (when vector y and z are in the 
same direction) and the correlation is 1. For the minimum correlation, we want the 
maximum angle between y and z, which is the case shown in Figure 3.2. 


If you still remember some trigonometry, 
all you need is that 


cos(20) = (cos 6)’ — (sin 0)’ 
= 0.8^ —0.6^ = 0.28 


Otherwise, you can solve the problem using 
Pythagoras's Theorem: 


0.8x1.221xh > h= 0.96 


z Ccos20 — 41^ — 0.96? = 0.28 


y 


0.6 0.6 
Figure 3.2 Minimum correlation and maximum angle between vectors y and z 
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QR decomposition 


QR decomposition: For each non-singular nxn matrix A, there is a unique pair of 
orthogonal matrix Q and upper-triangular matrix R with positive diagonal elements such 
that A- QR. ? 

QR decomposition is often used to solve linear systems Ax 2 b when A is a non-singular 
matrix. Since Q is an orthogonal matrix, Q'! = Q” and QRx = b — Rx = Q'b. Because R 
is an upper-triangular matrix, we can begin with x, (the equation is simply 


R, ,X, — (Q'b),), and recursively calculate all x,, Vi=n,n-1,---,1. 


If the programming language you are using does not have a function for the linear least 
squares regression, how would you design an algorithm to do so? 


Solution: The linear least squares regression is probably the most widely used statistical 
analysis method. Let's go over a standard approach to solving linear least squares 
regressions using matrices. A simple linear regression with n observations can be 
expressed as 

y, = ByX;g t BX ot Bux +6, Vi=l,---,n, where x, =1, Vi, is the intercept 


term and x,,,---,x, , , are p—1 exogenous regressors. 


The goal of the linear least squares regression is to find a set of 8 = [Ab s B 


that makes we the smallest. Let's express the linear regression in matrix format: 


i=] 
Y-Xf-e, where Y=[Y,Y,,---,Y,]’ and e-[&6,6,- 6,] are both nx1 column 
vectors; X is a nx p matrix with each column representing a regressor (including the 
intercept) and each row representing an observation. Then the problem becomes 


MRJUDC REL S =min(¥ -X p) (Y - X p) 


'? A nonsingular matrix Q is called an orthogonal matrix if Q ' » Q'. Q is orthogonal if and only if the 


columns (and rows) of Q form an orthonormal set of vectors in R”. The Gram-Schmidt 
orthonormalization process (often improved to increase numerical stability) is often used for QR 
decomposition. Please refer to a linear algebra textbook if you are interested in the Gram-Schmidt process. 
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To minimize the function f(f), taking the first derivative'' of f() with respect to £, 
we have f'(f)22X'(Y-X8)20— (X'X)B 2 X'Y, where (X'X) is a pxp 


symmetric matrix and X'Y isa pxl column vector. 


Let 4— (X' X) and b= X'Y, then the problem becomes AB = b, which can be solved 
using OR decomposition as we described. 


Alternatively, if the programming language has a function for matrix inverse, we can 
directly calculate B as A -(X^ X) XTY.” 
Since we are discussing linear regressions, it's worthwhile to point out the assumptions 


behind the linear least squares regression (a common statistics question at interviews): 
1. The relationship between Y and X is linear: Y= XJ +e. 


2. Elée|=0, Vi 2 1,--,n. 
3. var(£)-0^,i-1,-,n (constant variance), and E[e,e,] 2 0, i j (uncorrelated 
errors). 
4. No perfect multicollinearity: p(x,,x,)#+1,i#j7 where p(x,,x,) is the 
correlation of regressors x, and x;. 
5. & and x, are independent. 
Surely in practice, some of these assumptions are violated and the simple linear least 
squares regression is no longer the best linear unbiased estimator (BLUE). Many 


econometrics books dedicate significant chapters to addressing the effects of assumption 
violations and corresponding remedies. 


Determinant, eigenvalue and eigenvector 
Determinant: Let A be an nxn matrix with elements (A, n^ where i, j =1,---,n. The 


determinant of A is defined as a scalar: det(A) - 5 v(p)a,,a,,, a where 


p 
P=(P,; P» P,) is any permutation of (1, 2, ---, n); the sum is taken over all n! 


n,p,? 
possible permutations; and 


'' To do that, you do need a little knowledge about matrix derivatives. Some of the important derivative 


T T T 2. T 
equations for vectors/matrices are ða x xa m OAX — A, ae =(A’ 4 A)x, Ox AX 2A, 
Ox Ox Ox Ox OxOx. 
O(Ax + b) C(Dx +e 
sae ae = A C(Dx +e)+ D'C' (Ax +b), 
X 


? The matrix inverse introduces large numerical error if the matrix is close to singular or badly scaled. 
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us l, if p can be coverted to natural order by even number of exchanges 
ao -], if p can be coverted to natural order by odd number of exchanges - 
For example, determinants of 2x2 and 3x3 matrices can be calculated as 
p a b c 
zl “| —-ad-bc, det|| d e f||=aei+bfg+cdh-ceg-afh -bdi .” 
C 
g h i 


] 
det(A) 


Determinant properties: det( A" ) = det(A), det( AB) = det(A)det(B), det(A™) = 


Eigenvalue: Let 4 be an nxn matrix. A real number 4 is called an eigenvalue of A if 


there exists a nonzero vector x in R” such that Ax=Ax. Every nonzero vector x 
satisfying this equation is called an eigenvector of A associated with the eigenvalue 4. 


Eigenvalues and eigenvectors are crucial concepts in a variety of subjects such as 
ordinary differential equations, Markov chains, principal component analysis (PCA), etc. 
The importance of determinant lies in its relationship to eigenvalues/eigenvectors.! 


The determinant of matrix 4— AI, where 7 is an nxn identity matrix with ones on the 


main diagonal and zeros elsewhere, is called the characteristic polynomial of A. The 
equation det(A — AJ) =0 is called the characteristic equation of A. The eigenvalues of 


A are the real roots of the characteristic equation of A. Using the characteristic equation, 
we can also show that 4,4, --- A, =det(A) and » 4, = trace( A) = » A. 

i=l i-l 
A is diagonalizable if and only if it has linearly independent eigenvectors. '° Let 
A, 45,7, 4, be the eigenvalues of A, x, x,, ---, x, be the corresponding eigenvectors. 


and X =[x, |x, |--- | x, ], then 


A 


A, 


X'AX = = D > A = XDX `” > Æ = XD' X". 


3 In practice, determinant is usually not solved by the sum of all permutations because it is 
computationally inefficient. LU decomposition and cofactors are often used to calculate determinants 
instead. 

^ Determinant can also be applied to matrix inverse and linear equations as well. 

^ If all n eigenvalues are real and distinct, then the eigenvectors are independent and A is diagonalizable. 
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2 1 
If matrix A= f d , what are the eigenvalues and eigenvectors of A? 


Solution: This is a simple example of eigenvalues and eigenvectors. It can be solved 
using three related approaches: 


Approach A: Apply the definition of eigenvalues and eigenvectors directly. 


Xi 


Let A be an eigenvalue and x -| | be its corresponding eigenvector. By definition, we 


X5 
have 


2. T 2X 4X. Ax, 2x, +x, z: ÁX, 
Ax = = =Ax= = — 3(x,+x,) = A(x +x) 
L 2] x x +2x, Ax, X, * 2x, = AX, 
So either 4 =3, in which case x, = x, (plug 2 =3 into equation 2x, +x, = Ax, ) and the 


J2 
corresponding normalized eigenvector is | Jf , Or x, +x, =0, in which case the 
1/42 


42 
E P 


normalized eigenvector is | | and A=1 (plug x,--»x, into equation 


2x, * x, = Ax, ). 
Approach B: Use equation det(A — AI) = 0. 
det(A- 47) =0 > (2- 4)(2- 4)-120. Solving the equation, we have A4 =1 and 


À,-3. Applying the eigenvalues to Ax=Ax, we can get the corresponding 
eigenvectors. 


Approach C: Use equations 4, - 4, --- A4, =det(A) and 254 = trace(A) = DAs 


i=] i=l 


det(4)=2x2-1x1=3 and trace(A)=2x2=4. 


AxA =3 E A, =l 
A+A,=4) [A =3 


can get the corresponding eigenvectors. 


So we have Again apply the eigenvalues to Ax = Ax, and we 
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Positive semidefinite/definite matrix 


When A is a symmetric nxn matrix, as in the cases of covariance and correlation 
matrices, all the eigenvalues of A are real numbers. Furthermore, all eigenvectors that 
belong to distinct eigenvalues of A are orthogonal. 


Each of the following conditions is a necessary and sufficient condition to make a 
symmetric matrix A positive semidefinite: 


1. x’ Ax>0 for any nxl vector x. 

2. All eigenvalues of A are nonnegative. 

3. All the upper left (or lower right) submatrices A,, K =1,---, n have nonnegative 
determinants. '° 


Covariance/correlation matrices must also be positive semidefinite. If there is no perfect 
linear dependence among random variables, the covariance/correlation matrix must also 
be positive definite. Each of the following conditions is a necessary and sufficient 
condition to make a symmetric matrix A positive definite: 


1. x'Ax» 0 for any nonzero nx] vector x. 
2. All eigenvalues of A are positive. 


3. All the upper left (or lower right) submatrices A,, K =1,---, have positive 
determinants. 


There are 3 random variables x, y and z. The correlation between x and y is 0.8 and the 
correlation between x and z is 0.8. What is the maximum and minimum correlation 
between y and z? 


Solution: The problem can be solved using the positive semidefiniteness property of the 
correlation matrix. 


Let the correlation between y and z be p , then the correlation matrix for x, y and z is 


1 08 0.8 
P=/08 1 p 
08 p 1] 


'° A necessary, but not sufficient, condition for matrix A to be positive semidifinite is that A has no 
negative diagonal elements. 
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de) = Ixda | ^ osse (| | osaa? Hi 
p l p | | p 


= (I- p?)-0.8x(0.8—0.80)--0.8x(0.8p — 0.8) 2 -0.28-1.28p- p^ 20 
=> (p-1)(p—0.28) <0 => 0.28< pxl 


So the maximum correlation between y and z is 1, the minimum is 0.28. 


LU decomposition and Cholesky decomposition 


Let A be a nonsingular nxn matrix. LU decomposition expresses A as the product of a 
lower and upper triangular matrix: A = LU." 


LU decomposition can be use to solve Ax 2 b and calculate the determinant of A: 


LUx = b => Ux = y, Ly =b; det(A) = det(L) det(U) = | | L; | [U; ;. 


i=] j=l 


When A is a symmetric positive definite matrix, Cholesky decomposition expresses A 
as A — R' R, where R is a unique upper-triangular matrix with positive diagonal entries. 


Essentially, it is a LU decomposition with the property L =U”. 


Cholesky decomposition is useful in Monte Carlo simulation to generate correlated 
random variables as shown in the following problem: 


How do you generate two N(0,1) (standard normal distribution) random variables with 
correlation p if you have a random number generator for standard normal distribution? 


Solution: Two N(0,1) random variables x,, x, with a correlation p can be generated 
from independent N(0,1) random variables z,, z, using the following equations: 

X, =Z 

x, = pz +yl- p'z, 

It is easy to confirm that var(x,) = var(z,)=1, var(x,) = p^ var(z,) - (1— p^) var(z,) =1, 
and cov(x,,x,) =cov(z,, pz, + J1-pz,) —cov(z,pz)-p. 

This approach is a basic example using Cholesky decomposition to generate correlated 


random numbers. To generate correlated random variables that follow a n-dimensional 


'7 LU decomposition occurs naturally in Gaussian elimination. 
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multivariate normal distribution X =[X,,X,,---,X,]’ ~N(u,2) with mean 
H - [bru] and covariance matrix X (a nxn positive definite matrix), we can 
decompose the covariance matrix X into R’R and generate n independent N(0, 1) 
random variables z,, z,,---, z,. Let vector Z =[z,, z,,---,z,]’, then X can be generated 
as X = u+R’ Z.” 

Alternatively, X can also be generated using another important matrix decomposition 
called singular value decomposition (SVD): For any nx p matrix X, there exists a 


factorization of the form X =UDV", where U and V are nx p and px p orthogonal 


matrices, with columns of U spanning the column space of X, and the columns of V 
spanning the row space; D is a px p diagonal matrix called the singular values of X. 


For a positive definite covariance matrix, we have V =U and X-UDU'. Furthermore, 
D is the diagonal matrix of eigenvalues A, /5,--., 4, and U is the matrix of n 


corresponding eigenvectors. Let D^ be a diagonal matrix with diagonal elements 
NER NEN ey SEM then it is clear that D-(D'?^y -(D"^yD"y and 
X =UD"’(UD"’)’. Again, if we generate a vector of n independent N(0, 1) random 
variables Z -[z,, z,,--:,z,]’, X can be generated as X = u + (UD'^)Z. 


exp(- 1G AY E '(x- 40) 
(22) det(E) ^ 


? In general, if y = AX +b, where A and b are constant, then the covariance matrice £, = Ax 


'* The probability density of multivariate normal distribution is f(x) = 


A’ 


AY 


58 


Chapter 4 Probability Theory 


Chances are that you will face at least a couple of probability problems in most 
quantitative interviews. Probability theory is the foundation of every aspect of 
quantitative finance. As a result, it has become a popular topic in quantitative interviews. 


Although good intuition and logic can help you solve many of the probability problems, 
having a thorough understanding of basic probability theory will provide you with clear 
and concise solutions to most of the problems you are likely to encounter. Furthermore, 
probability theory is extremely valuable in explaining some of the seemingly- 
counterintuitive results. Armed with a little knowledge, you will find that many of the 
interview problems are no more than disguised textbook problems. 


So we dedicate this chapter to reviewing basic probability theory that 1s not only broadly 
tested in interviews but also likely to be helpful for your future career. ' The knowledge 
is applied to real interview problems to demonstrate the power of probability theory. 
Nevertheless, the necessity of knowledge in no way downplays the role of intuition and 
logic. Quite the contrary, common sense and sound judgment are always crucial for 
analyzing and solving either interview or real-life problems. As you will see in the 
following sections, all the techniques we discussed in Chapter 2 still play a vital role in 
solving many of the probability problems. 


Let’s have some fun playing the odds. 


4.1 Basic Probability Definitions and Set Operations 


First let's begin with some basic definitions and notations used in probability. These 
definitions and notations may seem dry without examples—which we will present 
momentarily—yet they are crucial to our understanding of probability theory. In 
addition, it will lay a solid ground for us to systematically approach probability 
problems. 


Outcome (o): the outcome of an experiment or trial. 


Sample space/Probability space (Q): the set of all possible outcomes of an experiment. 


' As I have emphasized in Chapter 3, this book does not teach probability or any other math topics due to 
the space limit—it is not my goal to do so, either. The book gives a summary of the frequently-tested 
knowledge and shows how it can be applied to a wide range of real interview problems. The knowledge 
used in this chapter is covered by most introductory probability books. It is always helpful to pick up one 
or two classic probability books in case you want to refresh your memory on some of the topics. My 
personal favorites are First Course in Probability by Sheldon Ross and Introduction to Probability by 
Dimitri P. Bertsekas and John N. Tsitsiklis. 


Probability Theory 


P(@): Probability of an outcome ( P(@) 2 0, Vo €Q, ` P(@) =1). 


oet 


Event: A set of outcomes and a subset of the sample space. 


P(A): Probability of an event A, P(A) = >. P(o). 


(€ A 
AUB: Union AUB is the set of outcomes in event A or in event B (or both). 


ACB or AB: Intersection 4AB (or AB ) is the set of outcomes in both A and B. 


A‘: The complement of A, which is the event “not A”. 


Mutually Exclusive: 47 B - 4 where © is an empty set. 


N N 
For any mutually exclusive events E,, E,,--- Ey, AU E = > P(E,). 
i=l 


i=] 


Random variable: A function that maps each outcome (œ) in the sample space (Q) into 
the set of real numbers. 


Let’s use the rolling of a six-sided dice to explain these definitions and notations. A roll 
of a dice has 6 possible outcomes (mapped to a random variable): 1, 2, 3, 4, 5, or 6. So 
the sample space Q is 11,2,3,4,5,6) and the probability of each outcome is 1/6 


(assuming a fair dice). We can define an event A representing the event that the outcome 
is an odd number 4 = {1, 3,5}, then the complement of A is A‘ = (2, 4, 6}. Clearly 
P(A)= P(1)+ P(3)+ P(5) 1/2. Let B be the event that the outcome is larger than 3: 
B={4,5,6}. Then the union is AUB = {1, 3,4,5,6} and the intersection is 
A B - (51. One popular random variable called indicator variable (a binary dummy 
variable) for event A is defined as the following: 


JL if xe tl, 3, 5} 
240, if xe fl, 3, 5} 
expected value of /, 1s E[/,]= P(A). 


F . Basically 7, =1 when A occurs and /, =0 if 4“ occurs. The 


Now, time for some examples. 
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Coin toss game 


Two gamblers are playing a coin toss game. Gambler A has (n +1) fair coins; B has n 


fair coins. What is the probability that A will have more heads than B if both flip all their 
coins? 


Solution: We have yet to cover all the powerful tools probability theory offers. What do 
we have now? Outcomes, events, event probabilities, and surely our reasoning 
capabilities! The one extra coin makes A different from B. If we remove a coin from 4A, 
A and B will become symmetric. Not surprisingly, the symmetry will give us a lot of 
nice properties. So let's remove the last coin of 4 and compare the number of heads in 
A's first n coins with B's n coins. There are three possible outcomes: 


E,: A's n coins have more heads than B's n coins; 
E,: A's n coins have equal number of heads as B's n coins; 


E, : A's n coins have fewer heads than B's n coins. 


By symmetry, the probability that 4 has more heads is equal to the probability that B has 
more heads. So we have P(E,) = P(E,). Let's denote P(E,) = P(E,) - xand P(E,) = y. 


Since > P(@)=1, we have 2x+ y «1. For event E,, A will always have more heads 


weQ 


than B no matter what A’s (n+1)th coin's side is; for event E,, A will have no more 
heads than B no matter what A's (n+1)th coin’s side is. For event E,, A’s (n Dh 
coin does make a difference. If it's a head, which happens with probability 0.5, it will 
make A have more heads than B. So the (n+1)th coin increases the probability that A 
has more heads than B by 0.5y and the total probability that 4 has more heads is 
x-0.5y2x40.5(1-2x) 20.5 when A has (n1) coins. 


Card game 


A casino offers a simple card game. There are 52 cards in a deck with 4 cards for each 
jack queen king ace 

value 2, 3, 4, 5,6, 7,8, 9, 10, J, Q, K. A. Each time the cards are thoroughly shuffled 

(so each card has equal probability of being selected). You pick up a card from the deck 

and the dealer picks another one without replacement. If you have a larger number, you 

win; if the numbers are equal or yours is smaller, the house wins—as in all other casinos, 

the house always has better odds of winning. What is your probability of winning? 


? Hint: What are the possible results (events) if we compare the number of heads in A's first n coins with 
B's n coins? By making the number of coins equal, we can take advantage of symmetry. For each event, 
what will happen if A’s last coin is a head? Or a tail? 
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Solution: One answer to this problem is to consider all 13 different outcomes of your 
card. The card can have a value 2,3, ---,A and each has 1/13 of probability. With a 


value of 2, the probability of winning is 0/51; with a value of 3, the probability of 
winning is 4/51 (when the dealer picks a 2); ...; with a value of A, the probability of 
winning is 48/51 (when the dealer picks a 2, 3, ---, or K). So your probability of 


winning is 


l ss u =] 4 4 12x13 8 


— x| —+—+4+---+— | =—— x (04+ 14+ -:-+12)= x =—., 
13 \51 51 51) 13x51 13x 51 2 17 

Although this is a straightforward solution and it elegantly uses the sum of an integer 
sequence, it is not the most efficient way to solve the problem. If you have got the core 
spirits of the coin tossing problem, you may approach the problem by considering three 
different outcomes: 


E, : Your card has a number larger than the dealer's; 
E, : Your card has a number equal to the dealer's; 


E, : Your card has a number lower than the dealer’s. 


Again by symmetry, P(E)- P(E). So we only need to figure out P(£,), the 


probability that two cards have equal value. Let's say you have randomly selected a card. 
Among the remaining 51 cards, only 3 cards will have the same value as your card. So 
the probability that the two cards have equal value is 3/51. As a result, the probability 


that you win is P(E,) -(1- P(E,))/2 =(1-3/51)/2 =8/17. 


Drunk passenger 


A line of 100 airline passengers are waiting to board a plane. They each hold a ticket to 
one of the 100 seats on that flight. For convenience, let's say that the n-th passenger in 
line has a ticket for the seat number n. Being drunk, the first person in line picks a 
random seat (equally likely for each seat). All of the other passengers are sober, and will 
go to their proper seats unless it is already occupied; In that case, they will randomly 
choose a free seat. You're person number 100. What is the probability that you end up 
in your seat (i.e., seat #100) ?° 


Solution: Let’s consider seats #1 and #100. There are two possible outcomes: 


? Hint: If you are trying to use complicated conditional probability to solve the problem, go back and think 
again. If you decide to start with a simpler version of the problem, starting with two passengers and 
increasing the number of passengers to show a pattern by induction, you can solve the problem more 
efficiently. But the problem is much simpler than that. Focus on events and symmetry and you will have 
an intuitive answer. 
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E, : Seat #1 is taken before #100; 
E,: Seat #100 is taken before #1. 


If any passenger takes seat #100 before #1 is taken, surely you will not end up in you 
own seat. But if any passenger takes #1 before #100 is taken, you will definitely end up 
in you own seat. By symmetry, either outcome has a probability of 0.5. So the 
probability that you end up in your seat is 50%. 


In case this over-simplified version of reasoning is not clear to you, consider the 
following detailed explanation: If the drunk passenger takes #1 by chance, then it’s clear 
all the rest of the passengers will have the correct seats. If he takes #100, then you will 
not get your seat. The probabilities that he takes #1 or #100 are equal. Otherwise assume 
that he takes the n-th seat, where n is a number between 2 and 99. Everyone between 2 
and (n—1) will get his own seat. That means the n-th passenger essentially becomes the 
new “drunk” guy with designated seat #1. If he chooses #1, all the rest of the passengers 
will have the correct seats. If he takes #100, then you will not get your seat. (The 
probabilities that he takes #1 or #100 are again equal.) Otherwise he will just make 
another passenger down the line the new “drunk” guy with designated seat #1 and each 
new “drunk” guy has equal probability of taking #1 or #100. Since at all jump points 
there's an equal probability for the “drunk” guy to choose seat #1 or 100, by symmetry, 
the probability that you, as the 1007/5 passenger, will seat in #100 is 0.5. 


N points on a circle 


Given N points drawn randomly on the circumference of a circle, what is the probability 
that they are all within a semicircle?^ 


Solution: Let's start at one point and clockwise label the points as 1,2,---,N. The 
probability that all the remaining N —1 points from 2 to N are in the clockwise 
semicircle starting at point 1 (That is, if point 1 is at 12:00, points 2 to N are all 
between 12:00 and 6:00) is 1/2" . Similarly the probability that a clockwise semicircle 
starting at any point i, where i € {2,---, N} contains all the other N —1 points is also 
1/25 

Claim: the events that all the other N —1 points are in the clockwise semicircle starting 
at point i, 7-21, 2,---, N are mutually exclusive. In other words, if we, starting at point i 
and proceeding clockwise along the circle, sequentially encounters points i+1, i+2,---, 
N,1,---, i-linhalfa circle, then starting at any other point j, we cannot encounter all 


* Hint: Consider the events that starting from a point n, you can reach all the rest of the points on the circle 
clockwise, n € {l,---, N] in a semicircle. Are these events mutually exclusive? 
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other points within a clockwise semicircle. Figure 4.1 clearly demonstrates this 
conclusion. If starting at point i and proceeding clockwise along the circle, we 
sequentially encounter points i+1,i+2,---, N,1,---,i—1 within half a circle, the 
clockwise arc between i—1 and i must be no less than half a circle. If we start at any 
other point, in order to reach all other points clockwise, the clockwise arc between i —1 
and i are always included. So we cannot reach all points within a clockwise semicircle 
starting from any other points. Hence, all these events are mutually exclusive and we 
have 


N N N 
Ps |= $res re, )- a ELA IM 


The same argument can be extended to any arcs that have a length less than half a circle. 
If the ratio of the arc length to the circumference of the circle is x (x €1/2), then the 


probability of all N points fitting into the arc is N xx". 


i 


Figure 4.1 N points fall in a clockwise semicircle starting from / 


4.2 Combinatorial Analysis 


Many problems in probability theory can be solved by simply counting the number of 
different ways that a certain event can occur. The mathematic theory of counting is 
often referred to as combinatorial analysis (or combinatorics). In this section, we will 
cover the basics of combinatorial analysis. 


Basic principle of counting: Let S be a set of length-k sequences. If there are 
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e n, possible first entries, 
e n, possible second entries for each first entry, 


e n, possible third entries for each combination of first and second entries, etc. 
Then there are a total of n, -n,---n, possible outcomes. 


Permutation: A rearrangement of objects into distinct sequence (1.e., order matters). 


n! ; ; 
Property: There are a different permutations of n objects, of which n, are 
n'n, !n,! 


alike, n, are alike, ---, n, are alike. 


Combination: An unordered collection of objects (i.e., order doesn’t matter). 


n ! 
Property: There are | \- - different combinations of n distinct objects taken 


r) (n—r)!r! 


r atatime. 
Binomial theorem: (x+ y)" = DNE y~ 
k=0 
Inclusion-Exclusion Principle: P(E, 0 E,) = P(E) + P(E,) - P(E E) 
P(E, VE, O E) P(E) + P(E,)+ P(E,) - P(E E) - P(E E,)-P(E,E,)+ P(E EE) 


and more generally, 


N 
P(E, UE, U...U Ey) - 9 P(E)- 9 P(E, E, te +(-1)™ 2- P(E, E, B) + 
i=] ij «b i «i^ «...i, 


ES (-1)^* P(E,E, U Ex) 


N 
where x P(E, E, +E, ) has | terms. 
r 


i «i, «...i, 


Poker hands 


Poker is a card game in which each player gets a hand of 5 cards. There are 52 cards in a 
deck. Each card has a value and belongs to a suit. There are 13 values, 


jack queen king ace spade club heart diamond 


2, 3, 4, 5, 6, 7, 8 9,10, J, Q, K, A,andfour suits, 4,%,9, e 
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What are the probabilities of getting hands with four-of-a-kind (four of the five cards 
with the same value)? Hands with a full house (three cards of one value and two cards of 
another value)? Hands with two pairs? 


Solution: The number of different hands of a five-card draw is the number of 5-element 
52 

subsets of a 52-element set, so total number of hands = |: f | = 2,598,960. 

Hands with a four-of-a-kind: First we can choose the value of the four cards with the 

same value, there are 13 choices. The 5th card can be any of the rest 48 cards (12 


choices for values and 4 choices for suits). So the number of hands with four-of-a kind 1s 
13x482 624. 


Hands with a Full House: In sequence we need to choose the value of the triple, 13 


4 
choices; the suits of the triple, HN choices; the value of the pair, 12 choices; and the 
4 l . 
suits of the pair, B choices. So the number of hands with full house is 


4 4 
iss (5 tas 3] =13x4x12x6=3,744. 


Hands with Two Pairs: In sequence we need to choose the values of the two pairs, 


13 4 4 
ra choices; the suits of the first pair, H choices; the suits of the second pair, K 


choices; and the remaining card, 44 (52—4x2, since the last cards can not have the 
same value as either pair) choices. So the number of hands with two pairs is 


13) (4) (4 
x} |x|. |x44 = 78x6x6x 44 = 123,552. 
2) (2) (2 


To calculate the probability of each, we only need to divide the number of hands of each 
kind by the total possible number of hands. 


Hopping rabbit 


A rabbit sits at the bottom of a staircase with n stairs. The rabbit can hop up only one or 
two stairs at a time. How many different ways are there for the rabbit to ascend to the 
top of the stairs?” 


` Hint: Consider an induction approach. Before the final hop to reach the n-th stair, the rabbit can be at 
either the (-1)th stair or the (”-2)th stair assuming n > 2. 
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Solution: Let’s begin with the simplest cases and consider solving the problem for any 
number of stairs using induction. For n=1, there is only one way and /(1)=1. For 


n=2, we can have one 2-stair hop or two 1-stair hops. So /(2)-2. For any n> 2, 


there are always two possibilities for the last hop, either it’s a 1-stair hop or a 2-stair hop. 
In the former case, the rabbit is at (n —1) before reaching n, and it has f(m—1) ways to 


reach (n—1). In the latter case, the rabbit is at (n—2) before reaching n, and it has 
f(n—-2) ways to reach (n—2). So we have f(n)= f(n—2)+ f(n—1). Using this 
function we can calculate f(n) for n —3, 4,---° 


Screwy pirates 2 


Having peacefully divided the loot (in chapter 2), the pirate team goes on for more 
looting and expands the group to 11 pirates. To protect their hard-won treasure, they 
gather together to put all the loot in a safe. Still being a democratic bunch, they decide 
that only a majority — any majority — of them (26) together can open the safe. So they 
ask a locksmith to put a certain number of locks on the safe. To access the treasure, 
every lock needs to be opened. Each lock can have multiple keys; but each key only 
opens one lock. The locksmith can give more than one key to each pirate. 


What is the smallest number of locks needed? And how many keys must each pirate 
carry? 


Solution: This problem is a good example of the application of combinatorial analysis in 
information sharing and cryptography. A general version of the problem was explained 
in a 1979 paper “How to Share a Secret" by Adi Shamir. Let's randomly select 5 pirates 
from the 1 1-member group; there must be a lock that none of them has the key to. Yet 
any of the other 6 pirates must have the key to this lock since any 6 pirates can open all 
locks. In other words, we must have a "special" lock to which none of the 5 selected 
pirates has a key and the other 6 pirates all have keys. Such 5-pirate groups are randomly 
selected. So for each combination of 5 pirates, there must be such a "special" lock. The 


11 | 
minimum number of locks needed is "n locks. Each lock has 6 keys, 
which are given to a unique 6-member subgroup. So each pirate must have 
462x 6 


— 252 keys. That's surely a lot of locks to put on a safe and a lot of keys for 


each pirate to carry. 


? You may have recognized that the sequence is a sequence of Fibonacci numbers. 
"Hint: every subgroup of 6 pirates should have the same key to a unique lock that the other 5 pirates do 
not have. 
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Chess tournament 


A chess tournament has 2" players with skills 1 > 2 > --- >2". It is organized as a 
knockout tournament, so that after each round only the winner proceeds to the next 
round. Except for the final, opponents in each round are drawn at random. Let’s also 
assume that when two players meet in a game, the player with better skills always wins. 
What's the probability that players 1 and 2 will meet in the final?? 


Solution: There are at least two approaches to solve the problem. The standard approach 
applies multiplication rule based on conditional probability, while a counting approach 
is far more efficient. (We will cover conditional probability in detail in the next section.) 


Let’s begin with the conditional probability approach, which is easier to grasp. Since 
there are 2” players, the tournament will have n rounds (including the final). For round 


1, players 2,3,---,2” each have probability to be 1’s rival, so the probability that 


2" — 
n n-] E 
] and 2 do not meet in round 1 is - ; = LG Condition on that 1 and 2 do not 


meet in round 1, 2"! players proceed to the 2nd round and the conditional probability 
27-2 2x(Q"*-1) 
yn] 7 9nd L] 
process until the (n — D£h round, in which there are 2^ (7-2" / 2"^) players left and the 
conditional probability that 1 and 2 will not meet in round (7-1) is 
27-2 2x(2*!-1) 

2-1 2-1 


Let E, be the event that 1 and 2 do not meet in round 1; 


that 1 and 2 will not meet in round 2 is . We can repeat the same 


E, be the event that 1 and 2 do not meet in rounds 1 and 2; 


E, , be the event that 1 and 2 do not meet in round 1,2,---,n—1. 


Apply the multiplication rule, we have 

P(1 and 2 meet in the nth game) = P(E,)x P(E, | E)x--xP(E, | BE, = E, ») 
n- . n-2 2-l n-l 

 2xQ I) 2x2 I) x 2xQ ) 2 


oF) ss Pf — 953] 


* Hint: Consider separating the players to two 2”” subgroups. What will happen if player 1 and 2 in the 
same group? Or not in the same group? 
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Now let’s move on to the counting approach. Figure 4.2A is the general case of what 
happens in the final. Player 1 always wins, so he will be in the final. From the figure, it 


is obvious that 2" players are separated to two 2”'-player subgroups and each group 


will have one player reaching the final. As shown in Figure 4.2B, for player 2 to reach 
the final, he/she must be in a different subgroup from 1. Since any of the remaining 


players in 2,3,---,2” are likely to be one of the (2"' — 1) players in the same subgroup 


as player 1 or one of the 2”' players in the subgroup different from player 1, the 
probability that 2 is in a different subgroup from 1 and that 1 and 2 will meet in the final 


n-] 


is simply : Clearly, the counting approach provides not only a simpler solution but 


2" — 


also more insight to the problem. 


General Case ] & 2 in the Final 


2™! players 2™! players 2™! players 2™! players 


A B 


Figure 4.2A The general case of separating 2" players into 2"'-player subgroups; 
4.2B The special case with players 1 and 2 in different groups 


Application letters 


You're sending job applications to 5 firms: Morgan Stanley, Lehman Brothers, UBS, 
Goldman Sachs, and Merrill Lynch. You have 5 envelopes on the table neatly typed with 
names and addresses of people at these 5 firms. You even have 5 cover letters 
personalized to each of these firms. Your 3-year-old tried to be helpful and stuffed each 
cover letter into each of the envelopes for you. Unfortunately she randomly put letters 
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into envelopes without realizing that the letters are personalized. What is the probability 
that all 5 cover letters are mailed to the wrong firms?” 


Solution: This problem is a classic example for the Inclusion-Exclusion Principle. In fact, 
a more general case is an example in Ross’ textbook First Course in Probability. 


Let's denote by £,,i=1,---,5 the event that the i-th letter has the correct envelope. Then 


5 
283 is the probability that at least one letter has the correct envelope and 


5 5 
l- 2 is the probability that all letters have the wrong envelopes. 2 can 


i=] i-l 


be calculated using the Inclusion-Exclusion Principle: 


P( o6 - Y P(E)- » P(E, E, ) ++ (CDI P(E E, --- E;) 


lh <h 


5 
It's obvious that P(E£,) = 2 Vi =1,---,5.So Y P(E) =]. 


i=] 
P(E, E,) is the event that both letter i, and letter i, have the correct envelope. The 


probability that i, has the correct envelope is 1/5; Conditioned on that i, has the correct 
envelope, the probability that i, has the correct envelope is 1/4 (there are only 4 


— 2)! 
envelopes left). So P(E, E, jii e 2d 
AE WS 5! 

5 5! 
There are = — — — members of P(E, E, ) in » PE, E, ), so we have 

2). 2YUS=2)! = "m ANS 

—2) | 

YE" OD 9 1 
"m Pos 5! 2 (52). 2 


similarly we have b3 P(E,E,E,) =. ^ P(E, E E, E,) - >> and 


| 
P(E E, E) = 


? Hint: The complement is that at least one letter is mailed to the correct firm. 
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s 1 l1 1 1 19 
i ae _j=l-—+—-—+—=— 
(vj 2r 3i 4! 5! 30 


i-l 


5 
So the probability that all 5 letters are mailed to the wrong firms is 1— p(w] = n 


Birthday problem 


How many people do we need in a class to make the probability that two people have the 
same birthday more than 1/2? (For simplicity, assume 365 days a year.) 


Solution: The number is surprisingly small: 23. Let’s say we have n people in the class. 
Without any restrictions, we have 365 possibilities for each individual’s birthday. The 


basic principle of counting tells us that there are 365” possible sequences. 


We want to find the number of those sequences that have no duplication of birthdays. 
For the first individual, we can choose any of the 365 days; but for the second, only 364 
remaining choices left, ..., for the rth individual, there are 365—r+1 choices. So for n 
people there are 365x364x---x(365-n+1) possible sequences where no two 


365x364x:::x(365-n+1) |, 


individuals have the same birthday. We need to have 365" 


for the odds to be in our favor. The smallest such n is 23. 


100th digit 


What is the 100th digit to the right of the decimal point in the decimal representation of 
(15:2) 9 910 


Solution: If you still have not figure out the solution from the hint, here is one more hint: 
(1+ V2)" +(1- J2y! is an integer when n = 3000. 


Applying the binomial theorem for (x + y)" , we have 


n n 4 n : ý 
(1+V2)" = n 2-2 "x uim un 
Lco k=2j,0sjs% k=2j+10S j«7 


° Hint: (1+ J2y +(1- J2y = 6. What will happen to (1— J2y" as n becomes large? 
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"(n k h n 2 n 
agon £ pes- [jn 
AG k=2j,0s j<? k=2 j+1,0s j<” 

2 2 
So (1 #2)" +(1- V2)" E » Ha NM which is always an integer. It is easy to 


kz2j,0s js 
JUS] 7 


see that 0<(1—- V2) <<10™%, So the 100th digit of (1+ 2)" must be 9. 


Cubic of integer 


Let x be an integer between 1 and 10", what is the probability that the cubic of x ends 
with 11?" 


Solution: All integers can be expressed as x 2 a 10b, where a is the last digit of x. 
Applying the binomial theorem, we have x? = (a + 10b)! = a’ 30a? b - 300ab? + 10000’. 


The unit digit of x? only depends on a’. So a? has a unit digit of 1. Only a =1 satisfies 
this requirement and a° =1. Since a’ =1, the tenth digit only depends on 30a?b = 30b. 


So we must have that 3b ends in 1, which requires the last digit of b to be 7. 
Consequently, the last two digits of x should be 71, which has a probability of 1% for 
integers between 1 and 10". 


4.3 Conditional Probability and Bayes' formula 


Many financial transactions are responses to probability adjustments based on new—and 
most likely incomplete—information. Conditional probability surely is one of the most 
popular test subjects in quantitative interviews. So in this section, we focus on basic 
conditional probability definitions and theorems. 


P(AB) 
P(B) 


Conditional probability P(A| B): If P(B)»0, then P(A4| B)- is the fraction 


of B outcomes that are also 4 outcomes. 


Multiplication Rule: P(E E, --- E,) = P(E))P(E, | E)P(E, | E,E,)--- P(E, | E + Epa). 


'' Hint: The last two digits of x only depend on the last two digits of x. 
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Law of total probability: for any mutually exclusive events (EI , 1=1,2,---,n, whose 


union is the entire sample space ( F; O F, =, Vi + j; UF =Q), we have 


i=] 


P(E)= P(EF,) + P(EF,) +-+ P(EF,) = X P(E | F)P(F) 


i=] 


= P(E | F,)P(F,) + P(E | Fj) P(E) ++ P(E | F PCF, ) 
Independent events: P(EF) = P(E)P(F) => P(EF‘)=P(E)P(F‘). 
Independence is a symmetric relation: X is independent of Y <> Y is independent of X. 
P(E |F PF) 


Bayes’ Formula: P(F, | E) = — if E, i = 1, ---, n, are mutually 
Y P(E | F,)P(F,) 
i=] 


l 


exclusive events whose union is the entire sample space. 


As the following examples will demonstrate, not all conditional probability problems 
have intuitive solutions. Many demand logical analysis instead. 


Boys and girls 
Part A. A company is holding a dinner for working mothers with at least one son. Ms. 


Jackson, a mother with two children, is invited. What is the probability that both 
children are boys? 


Solution: The sample space of two children is given by Q = {(b, b), (b, g), (,b), (2,2) 
(e.g.,(g,5) means the older child is a girl and the younger child a boy), and each 
outcome has the same probability. Since Ms. Jackson is invited, she has at least one son. 
Let B be the event that at least one of the children is a boy and A be the event that both 
children are boys, we have 


P(A} By = PAB) PUOL) A 1! 
= P(B)  P(((bb),(bg)(g,b)) 3/4 3. 


Part B. Your new colleague, Ms. Parker is known to have two children. If you see her 
walking with one of her children and that child is a boy, what is the probability that both 
children are boys? 
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Solution: the other child is equally likely to be a boy or a girl (independent of the boy 
you’ve seen), so the probability that both children are boys is 1/2. 


Notice the subtle difference between part A and part B. In part A, the problem essentially 
asks given there is at least one boy in two children, what is the conditional probability 
that both children are boys. Part B asks that given one child is a boy, what is the 
conditional probability that the other child is also a boy. For both parts, we need to 
assume that each child is equal likely to be a boy or a girl. 


All-girl world? 


In a primitive society, every couple prefers to have a baby girl. There is a 5096 chance 
that each child they have is a girl, and the genders of their children are mutually 
independent. If each couple insists on having more children until they get a girl and once 
they have a girl they will stop having more children, what will eventually happen to the 
fraction of girls in this society? 


Solution: It was surprising that many interviewees—include many who studied 
probability—have the misconception that there will be more girls. Do not let the word 
"prefer" and a wrong intuition misguide you. The fraction of baby girls are driven by 
nature, or at least the X and Y chromosomes, not by the couples' preference. You only 
need to look at the key information: 50% and independence. Every new-born child has 
equal probability of being a boy or a girl regardless of the gender of any other children. 
So the fraction of girls born is always 50% and the fractions of girls in the society will 
stay stable at 50%. 


Unfair coin 


You are given 1000 coins. Among them, 1 coin has heads on both sides. The other 999 
coins are fair coins. You randomly choose a coin and toss it 10 times. Each time, the 
coin turns up heads. What is the probability that the coin you choose is the unfair one? 


Solution: This 1s a classic conditional probability question that uses Bayes' theorem. Let 
A be the event that the chosen coin is the unfair one, then A‘ is the event that the chosen 
coin is a fair one. Let B be the event that all ten tosses turn up heads. Apply Bayes’ 
P(B|A)P(4) _ P(B | A)P(A) 


theorem we have P(A | B) = P(B) = P(B| A)P(A) + P(B | A) P(A) ; 


The priors are P(4)=1/1000 and P(4°)=999/1000. If the coin is unfair, it always 
turns up heads, so P(B | A) =1. If the coin is fair, each time it has 1/2 probability turning 
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up heads. So P(B| 4^) - (1/2)? =1/1024. Plug in all the available information and we 
have the answer: 


P(B | A)P(A) E 1/1000x1 


12107 E REE E SL MM E A MUT 
P(B| A)P(A) + P(B| A)P(4') | 1/1000x14- 999/1000 x 1/1024 


Fair probability from an unfair coin 


If you have an unfair coin, which may bias toward either heads or tails at an unknown 
probability, can you generate even odds using this coin? 


Solution: Unlike fair coins, we clearly can not generate even odds with one toss using an 
unfair coin. How about using 2 tosses? Let p, be the probability the coin will yield 


head, and p, be the probability the coin will yield tails ( p,, + p, 21). Consider two 


independent tosses. We have four possible outcomes HH, HT, TH and TT with 
probabilities P(HH)= p,p,, P(HT)= pyp,, P(TH)- p,p,, and P(TT)= p,p,. 


So we have P(HT) - P(TH). By assigning HT to winning and TH to losing, we can 
generate even odds." 


Dart game 


Jason throws two darts at a dartboard, aiming for the center. The second dart lands 
farther from the center than the first. If Jason throws a third dart aiming for the center, 
what is the probability that the third throw is farther from the center than the first? 
Assume Jason's skillfulness is constant. 


Solution: A standard answer directly applies the conditional probability by enumerating 
all possible outcomes. If we rank the three darts' results from the best (A) to the worst 
(C), there are 6 possible outcomes with equal probability: 


'7 | should point out that this simple approach is not the most efficient approach since I am disregarding 
the cases HH and TT. When the coin has high bias (one side is far more likely than the other side to occur), 
the method may take many runs to generate one useful result. For more complex algorithm that increasing 
efficiency, please refer to Tree Algorithms for Unbiased Coin Tossing with a Biased Coin by Quentin F. 
Stout and Bette L. Warren, Annals of Probability 12 (1984), pp. 212-222. 
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Outcome l 2 3 4 5 6 

lstthrow A B A C B C 
2nd throw B A C A C B 
3rdthrow C C B B A A 


The information from the first two throws eliminates outcomes 2, 4 and 6. Conditioned 
on outcomes 1, 3, and 5, the outcomes that the 3rd throw is worse than the 1st throw are 
outcomes | and 3. So there is 2/3 probability that the third throw is farther from the 
center than the first. 


This approach surely 1s reasonable. Nevertheless, it is not an efficient approach. When 
the number of darts is small, we can easily enumerate all outcomes. What if it is a more 
complex version of the original problem: 


Jason throws n (n 2 5) darts at a dartboard, aiming for the center. Each subsequent dart 
is farther from the center than the first dart. If Jason throws the (n+1)th dart, what is the 
probability that it is also farther from the center than his first? 


This question is equivalent to a simple question: what is the probability that the (n + 1)th 
throw is not the best among all (n +1) throws? Since the Ist throw is the best among the 
first n throws, essentially I am saying the event that the (n - 1)th throw is the best of all 
(n+1) throws (let's call it 4, 
of the first n throws (let's call it 4 ). In fact, A, 


n throws. Are these two events really independent? The answer is a resounding yes. If it 
is not obvious to you that A, is independent of the order of the first n throws, let's look 


) is independent of the event that the 1st throw is the best 
is independent of the order of the first 


+] 


n+] 
at it another way: the order of the first n throws is independent of 4, , . Surely this claim 
is conspicuous. But independence is symmetric! Since the probability of A,,, is 
| /(n +1), the probability that (n + Dh throw is not the best is n/(n +1).'° 

For the original version, three darts are thrown independently, they each have a 1/3 
chance of being the best throw. As long as the third dart is not the best throw, it will be 
worse than the first dart. Therefore the answer is 2/3. 


Birthday line 


At a movie theater, a whimsical manager announces that she will give a free ticket to the 
first person in line whose birthday is the same as someone who has already bought a 
ticket. You are given the opportunity to choose any position in line. Assuming that you 


3 Here you can again use symmetry argument: each throw is equally likely to be the best. 
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don't know anyone else's birthday and all birthdays are distributed randomly throughout 
the year (assuming 365 da ays in a year), what position in line gives you the largest chance 
of getting the free ticket?’ 


Solution: If you have solved the problem that no two people have the same birthday in 
an n-people group, this new problem is just a small extension. Assume that you choose 
to be the n-th person in line. In order for you to get the free ticket, all of the first n—1 
individuals in line must have different birthdays and your birthday needs to be the same 
as one of those n—1 individuals. 


p(n) = p( first n —1 people have no same birthday) x p( yours among those n —1 birthdays) 


. 365x364 x--.(365— n+2) n-l 
365" 7365 


It is intuitive to argue that when n is small, increasing n will increase your chance of 
getting the free ticket since the increase of p(yours among those n —1 birthdays) is 
more significant than the decrease in p( first n—1 people have no same birthday). So 
when n is small, we have P(n) > P(n-1). As n increases, gradually the negative impact 
of p(first n—1 peoplehaveno same birthday ) will catch up and at a certain point we 
will have P(n-1) « P(n). So we need to find such an n that satisfies P(n) > P(n-1) 
and P(n)> P(n41). 


365. 364 365—(n-3) n-2 
XX —————— X ——————— 


P(n-1)- 
305" 365 365 365 
P(n)= 365. 364. iy DOP T= 2) n-l 
365 365 365 365 
pins- 305 5 364 sony 365-(0-2), 365-(n-) n 
365 365 365 365 365 


365-(n-2) n-1. n-2 


P(n) > P(n-1) > : 
-3n - 363 « 0 
Hence, xm " 365 s" n [oem 
PoEPundgs4-5.299UP-. m | gene36530 
365 365 365 


You should be the 20th person in line. 


^ Hint: If you are the n-th person in line, to get the free ticket, the first (n-1) people in line must not have 
the same birthday and you must have the same birthday as one of them. 


77 


Probability Theory 


Dice order 


We throw 3 dice one by one. What is the probability that we obtain 3 points in strictly 
increasing order?" 


Solution: To have 3 points in strictly increasing order, first all three points must be 
different numbers. Conditioned on three different numbers, the probability of strictly 
increasing order is simply 1/3!=1/6 (one specific sequence out of all possible 
permutations). So we have 


P = P(different numbers in all three throws) x P(increasing order|3 different numbers) 
= (1x2x4)x1 2 5/54 | 


Monty Hall problem 


Monty Hall problem is a probability puzzle based on an old American show Let's Make 
a Deal. The problem is named after the show's host. Suppose you're on the show now, 
and you're given the choice of 3 doors. Behind one door is a car; behind the other two, 
goats. You don't know ahead of time what is behind each of the doors. 


You pick one of the doors and announce it. As soon as you pick the door, Monty opens 
one of the other two doors that he knows has a goat behind it. Then he gives you the 
option to either keep your original choice or switch to the third door. Should you switch? 
What is the probability of winning a car if you switch? 


Solution: If you don't switch, whether you win or not is independent of Monty's action 
of showing you a goat, so your probability of winning is 1/3. What if you switch? Many 
would argue that since there are only two doors left after Monty shows a door with goat, 
the probability of winning is 1/2. But is this argument correct? 


If you look at the problem from a different perspective, the answer becomes clear. Using 
a switching strategy, you win the car if and only if you originally pick a door with a goat, 
which has a probability of 2/3 (You pick a door with a goat, Monty shows a door with 
another goat, so the one you switch to must have a car behind it). If you originally 
picked the door with the car, which has a probability of 1/3, you will lose by switching. 
So your probability of winning by switching is actually 2/3. 


^ Hint: To obtain 3 points in strictly increasing order, the 3 points must be different. For 3 different points 
In a sequence, strictly increasing order is one of the possible permutations. 
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Amoeba population 


There is a one amoeba in a pond. After every minute the amoeba may die, stay the same, 
split into two or split into three with equal probability. All its offspring, if it has any, will 
behave the same (and independent of other amoebas). What is the probability the 
amoeba population will die out? 


Solution: This is just another standard conditional probability problem once you realize 
we need to derive the probability conditioned on what happens to the amoeba one 
minute later. Let P(E) be the probability that the amoeba population will die out and 


apply the law of total probability conditioned on what happens to the amoeba one 
minute later: 


P(E) = P(E| F)P(F) € (E| E)P(F,) € P(E| F,)P(F.) . 

For the original amoeba, as stated in the question, there are four possible mutually 
exclusive events each with probability 1/4. Let's denote F, as the event the amoeba dies; 
F, as the event that it stays the same; F} as the event that it splits into two; F, as the 
event that it splits into three. For event F, P(E|Fj)-1 since no amoeba is left. 


P(E|F,)- P(E) since the state is the same as the beginning. For F}, there are two 


amoebas; either behaves the same as the original one. The total amoeba population will 
die only if both amoebas die out. Since they are independent, the probability that they 


both will die out is P(E)'. Similarly we have P(F,) = P(E)’. Plug in all the numbers, 
the equation becomes P(E)- 1/4x1+1/4x P(E) -1/4x P(E) +1/4x P(E). Solve 
this equation with the restriction 0 < P(E) «1, and we will get P(E) = 42-12 0.414 
(The other two roots of the equation are 1 and sJ =1 ). 


Candies in a jar 


You are taking out candies one by one from a jar that has 10 red candies, 20 blue candies, 
and 30 green candies in it. What is the probability that there are at least 1 blue candy and 
1 green candy left in the jar when you have taken out all the red candies?'° 


Solution: At first look, this problem appears to be a combinatorial one. However, a 
conditional probability approach gives a much more intuitive answer. Let T,, T, and T, 


'? Hint: If there are at least 1 blue candy and 1 green candy left, the last red candy must have been 
removed before the last blue candy and the last green candy in the sequence of 60 candies. What is the 
probability that the blue candy is the last one in the 60-candy sequence? Conditioned on that, what is the 
probability that the last green candy is the last one in the 30-candy sequence (10 red, 20 green)? What if 
the green candy is the last one in the 60-candy sequence? 
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be the number that the last red, blue, and green candies are taken out respectively. To 
have at least 1 blue candy and 1 green candy left when all the red candies are taken out, 
we need to have T, «T, and T,«T,. In other words, we want to derive 


PT, « T, AT, « T, ). There are two mutually exclusive events that satisfy T, < T, and 
T, «T: T; «T, « T, and T; <T, « T. 

- P(T, «T, oT, < T,)= P(T, «T, < T,)+ P(T, < T, < T,) 

T, « T, « T, means that the last candy is green (T, = 60). Since each of the 60 candies 
are equally likely to be the last candy and among them 30 are green ones, we have 


P(T, = 60) -5 Conditioned on T, =60, we need P(T, « T, |T, = 60). Among the 30 


red and blue candies, each candy is again equally likely to be the last candy and there are 


20 blue candies, so P(T, « T, |T, = 60) -2 and P(T <T,<T DIT Similarly, 
we have P(T, « T, P Dade i 
60 40 


Hence, 


P(T, <T, ^T, « T,) = P(T. <T, «T.)  P(T, <T, <T,) = A A 
60 30 60 40 12 


Coin toss game 


Two players, A and B, alternatively toss a fair coin (A tosses the coin first, then B tosses 
the coin, then 4, then B...). The sequence of heads and tails is recorded. If there is a 
head followed by a tail (HT subsequence), the game ends and the person who tosses the 
tail wins. What is the probability that 4 wins the game?! 


Solution: Let P(A) be the probability that A wins; then the probability that B wins is 
P(B)=1- P(A). Let's condition P(A) on A's first toss, which has 1/2 probability of H 
(heads) and 1/2 probability of T (tails). 


P(A) =1/2P(A| H) -1/2P(A| T) 


If A's first toss is T, then B essentially becomes the first to toss (An H is required for the 
HT subsequence). So we have P(A|T) - P(B) =1-— P(A). 


If A’s first toss ends in H, let's further condition on B’s first toss. B has 1/2 probability 
of getting T, in that case A loses. For the 1/2 probability that B gets H, B essentially 


'7 Hint: condition on the result of A's first toss and use symmetry. 
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becomes the first one to toss an H. In that case, A has (1- P(A|H )) probability of 
winning. So P(A|H) 21/2x0*1/2(1— P(A4|H)) => P(A|H) 21/3 


Combining all the available information, we have 
P(A) 1/2x1/341/2(1— P(A)) => P(A) - 4/9. 


Sanity check: we can see that P(A) «1/2 , which is reasonable since A cannot win in his 
first toss, yet B has 1/4 probability to win in her first toss. 


Russian roulette series 


Let's play a traditional version of Russian roulette. A single bullet is put into a 6- 
chamber revolver. The barrel is randomly spun so that each chamber is equally likely to 
be under the hammer. Two players take turns to pull the trigger—with the gun 
unfortunately pointing at one's own head—without further spinning until the gun goes 
off and the person who gets killed loses. If you, one of the players, can choose to go first 
or second, how will you choose? And what is your probability of loss? 


Solution: Many people have the wrong impression that the first person has higher 
probability of loss. After all, the first player has a 1/6 chance of getting killed in the first 
round before the second player starts. Unfortunately, this is one of the few times that 
intuition is wrong. Once the barrel is spun, the position of the bullet is fixed. If you go 
first, you lose if and only if the bullet is in chamber 1, 3 and 5. So the probability that 
you lose is the same as the second player, 1/2. [n that sense, whether to go first or second 
does not matter. 


Now, let's change the rule slightly. We will spin the barrel again after every trigger pull. 
Will you choose to be the first or the second player? And what is your probability of loss? 


Solution: The difference is that each run now becomes independent. Assume that the 
first player's probability of losing is p, then the second player's probability of losing is 
1— p. Let's condition the probability on the first person's first trigger pull. He has 1/6 
probability of losing in this run. Otherwise, he essentially becomes the second player in 
the game with new (conditional) probability of losing 1— p. That happens with 
probability 5/6. That gives us p=1x1/6+(l- p)x5/6— p=6/11. So you should 
choose to be the second player and have 5/11 probability of losing. 


If instead of one bullet, two bullets are randomly put in the chamber. Your opponent 


played the first and he was alive after the first trigger pull. You are given the option 
whether to spin the barrel. Should you spin the barrel? 
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Solution: if you spin the barrel, the probability that you will lose in this round is 2/6. If 
you don’t spin the barrel, there are only 5 chambers left and your probability of losing in 
this round (conditioned on that your opponent survived) is 2/5. So you should spin the 
barrel. 


What if the two bullets are randomly put in two consecutive positions? If your opponent 
survived his first round, should you spin the barrel? 


Solution: Now we have to condition our probability on the fact that the positions of the 
two bullets are consecutive. As shown in Figure 4.3, let’s label the empty chambers as 1, 
2, 3 and 4; label the ones with bullets 5 and 6. Since your opponent survived the first 
round, the possible position he encountered is 1, 2, 3 or 4 with equal probability. With 
1/4 chance, the next one is a bullet (the position was 4). So if you don’t spin, the chance 
of survival is 3/4. If you spin the barrel, each position has equal probability of being 
chosen, and your chance of survival is only 2/3. So you should not spin the barrel. 


Figure 4.3 Russian roulette with two consecutive bullets. 


Aces 


Fifty-two cards are randomly distributed to 4 players with each player getting 13 cards. 
What is the probability that each of them will have an ace? 


Solution: The problem can be answered using standard counting methods. To distribute 
52! 


PETENTEM permutations. If each player 


52 cards to 4 players with 13 cards each has 


82 


A Practical Guide To Quantitative Finance Interviews 


needs to have one ace, we can distribute the aces first, which has 4! ways. Then we 
48! 


distribute the rest 48 cards to 4 players with 12 cards each, which has ——————— 
12!12!12112! 


permutations. So the probability that each of them will have an Ace is 
48! 52! 32; 39. 26. 13 


RR, ee ey ict eai 
; 12121212! 131131313! - 52. 5] "50 49 
The logic becomes clearer if we use a conditional probability approach. Let’s begin with 
any one of the four aces; it has probability 52/52 =1 of belonging to a pile. The second 
ace can be any of the remaining 51 cards, among which 39 belong to a pile different 
from the first ace. So the probability that the second ace is not in the pile of the first ace 
is 39/51. Now there are 50 cards left, among which 26 belong to the other two piles. So 
the conditional probability that the third ace is in one of the other 2 piles given the first 
two aces are already in different piles is 26/50. Similarly, the conditional probability 
that the fourth ace is in the pile different from the first three aces given that the first 
three aces are in different piles is 13/49. So the probability that each pile has an ace is 
39 26 13 


xX — X — X —R 


5] 50 49` 


Gambler's ruin problem 


A gambler starts with an initial fortune of i dollars. On each successive game, the 
gambler wins $1 with probability p, 0 « p «1, or loses $1 with probability q 21— p. He 


will stop if he either accumulates N dollars or loses all his money. What is the 
probability that he will end up with N dollars? 


Solution: This is a classic textbook probability problem called the Gambler's Ruin 
Problem. Interestingly, it is still widely used in quantitative interviews. 


From any initial state i (the dollars the gambler has), 0x i € N, let P be the probability 


that the gambler's fortune will reach N instead of 0. The next state is either i+ 1 with 
probability p or i —1 with probability q. So we have 


2 i 
R = pRa + QF > Enah S= $a-no-[£ dh. (£] (f - f) 


We also have the boundary probabilities P, 2 0 and P, =1. 


So starting from P,, we can successively evaluate P as an expression of P: 
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P = pP, «qP, > P,=—P -«£|a 
P 


p 
2 
P, = (2) p 
P XP 


Extending this expression to P,, we have 


E N 
-— 2: EUD T ES 
peta | tee | ges Gp 

p p 


NH, if q/p-l 
l-q/p_, l-(q/p)' p , 
—_—. ifq/p#1 —-——_P,, if p#1/2 
=> P= 51-(q/ p)“ died => P=41-(q/p)” ` de 
1/N, if q/ p=) i/ N, if p-1/2 


Basketball scores 


A basketball player is taking 100 free throws. She scores one point if the ball passes 
through the hoop and zero point if she misses. She has scored on her first throw and 
missed on her second. For each of the following throw the probability of her scoring is 
the fraction of throws she has made so far. For example, if she has scored 23 points after 
the 40th throw, the probability that she will score in the 41th throw is 23/40. After 100 
throws (including the first and the second), what is the probability that she scores exactly 
50 baskets?'® 


Solution: Let (n,k), 1<k<n, be the event that the player scores k baskets after n 
throws and P,, = P((n,k)). The solution is surprisingly simple if we use an induction 


approach starting with n 23. The third throw has 1/2 probability of scoring. So we have 
P,,=1/2 and P,,-1/2. For the case when n=4, let's apply the law of total 


probability 


'* Hint: Again, do not let the number 100 scares you. Start with smallest n, solve the problem; try to find a 
pattern by increasing n; and prove the pattern using induction. 
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P, = P((4,1)1G,D) x By + P(4,013,) x Ba = 5x5 + 0X5 = 
P, = P((4,2)|(3,D)* B, + P((4,2) |(3,2)) x B, 1x14 141.1 
323 2 3 
P, = P((4,3163,D)* P, P(4,31G,2)x Ba = 0x5 +x x= : 


The results indicate that P, , -—, Vk =1,2,---,2—1, and give the hint that the law of 
En 


total probability can be used in the induction step. 


Induction step: given that P,, - Vk =1, 2,---,n-1, we need to prove 
n— 


IE — — n Vk -1,2,-., n. To show it, simply apply the law of total 
(n-l)-]1 n 

probability: 

P, = P(miss |(n, k)) P,, + P(score| (nk 1)) P, , 


njn-l n n-l n 


(i-i ++ QE E 


The equation is also applicable to the P,,, and P although in these cases pon -0 
n 


n+l, fps 


n -1 
Hence, Pooso = 1/99 . 


and (1-4)=0. respectively. So we have P,, EUM Vk =1,2,---,n—land Vn22. 
n 


Cars on road 


If the probability of observing at least one car on a highway during any 20-minute time 
interval is 609/625, then what is the probability of observing at least one car during any 
5-minute time interval? Assume that the probability of seeing a car at any moment is 
uniform (constant) for the entire 20 minutes. 


Solution: We can break down the 20-minute interval into a sequence of 4 non- 
overlapping 5-minute intervals. Because of constant default probability (of observing a 
car), the probability of observing a car in any 5-minute interval is constant. Let’s denote 
the probability to be p, then the probability that in any 5-minute interval we do not 
observe a car is 1— p. 


85 


Probability Theory 


The probability that we do not observe any car in all four of such independent 5-minute 
intervals is (1— p)* =1—609/625 =16/625, which gives p =3/5. 


4.4 Discrete and Continuous Distributions 


In this section, we review a variety of distribution functions for random variables that 
are widely used in quantitative modeling. Although it may not be necessary to memorize 
the properties of these distributions, having an intuitive understanding of the 
distributions and having the ability to quickly derive important properties are valuable 
skills in practice. As usual, let’s begin with the theories: 

Common function of random variables 


Table 4.1 summarizes how the basic properties of discrete and continuous random 
variables are defined or calculated. These are the basics you should commit to memory. 


Random variable 


Cumulative distribution function/cdf | F(a)- P(X <a} | F(a)= [^ f(x)de 
Probability mass function /pmf pmf. d 
i _ df: f(x) =— F(x) 
Probability density function /pdf p(x)= P(X =x} PE SOO =F 
Expected value/ E[ X] pa Aid (x) [ xf (x)dx 
x: p(x)> —00 


Expected value of g(X)/ Ele(X)] p BOPG) OGL 
EIX - EXD? = EIX? -EXI 


Standard deviation of X/ std (X) Jvar(X) 


Table 4.1 Basic properties of discrete and continuous random variables 


Discrete random variables 


Table 4.2 includes some of the most widely-used discrete distributions. Discrete uniform 
random variable represents the occurrence of a value between number a and b when all 


values in the set fa, at+l,---, b} have equal probability. Binomial random variable 
represents the number of successes in a sequence of n experiments when each trial is 


/? For continuous random variables, P(X = x) O0, Vx e (—00,00), so P{X < x) - P{X <x}. 
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independently a success with probability p. Poisson random variable represents the 
number of events occurring in a fixed period of time with the expected number of 
occurrences At when events occur with a known average rate A4 and are independent of 
the time since the last event. Geometric random variable represents the trial number (n) 
to get the first success when each trial is independently a success with probability p. 
Negative Binomial random variable represents the trial number to get to the r-th success 
when each trial is independently a success with probability p. 


EE i 
m x! , — MV» 
P(x) - 0-7 p) p, x-L,2,- 


Negative ECCL m i 
Binomial Pa - [11s (I- p) , x=r,r+l, 


Table 4.2 Probability mass function, expected value and variance of discrete random 
variables 


Continuous random variables 


Table 4.3 includes some of the commonly encountered continuous distributions. 
Uniform distribution describes a random variable uniformly distributed over the interval 
[a,b]. Because of the central limit theorem, normal distribution/Gaussian distribution is 
by far the most popular continuous distribution. Exponential distribution models the 
arrival time of an event if it has a constant arrival rate 4. Gamma distribution with 
parameters (a, A) often arises, in practice, as the distribution of the amount of time one 
has to wait until a total of n events occur. Beta distributions are used to model events 


20 Here we use the product of arrival rate A and time ¢ to define the parameter (expected value) since it is 
the definition used in many Poisson process studies. 
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that are constrained within a defined interval. By adjusting the shape parameters a and p, 
it can model different shapes of probability density functions.' 


Probability density function (pdf) E[X] 


— —aq 


"ZU OP HN 79 ^" NN 


e ^ a-l 
I(a*f) ang A, 8 | aw o 
L E ———À EI 


Table 4.3 Probability density function, expected value and variance of continuous 
random variables 


Normal 


Meeting probability 


Two bankers each arrive at the station at some random time between 5:00 am and 6:00 
am (arrival time for either banker is uniformly distributed). They stay exactly five 
minutes and then leave. What is the probability they will meet on a given day? 


Solution: Assume banker A arrives X minutes after 5:00 am and B arrives Y minutes after 
5:00 am. X and Y are independent uniform distribution between 0 and 60. Since both 
only stay exactly five minutes, as shown in Figure 4.4, A and B meet if and only if 
|X -—Y |s5. 


So the probability that A and B will meet is simply the area of the shadowed region 
divided by the area of the square (the rest of the region can be combined to a square with 


| 0x60—-2x(1/2x55x55) (60--55)x(60—55) 23 
size length 55): E - —— Tv 


?' For example, beta distribution is widely used in modeling loss given default in risk management. If you 
are familiar with Bayesian statistics, you will also recognize it as a popular conjugate prior function. 


88 


A Practical Guide To Quantitative Finance Interviews 


601—————— ZH 
ar eet. E: 
y P 
A 
E. 3 
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4 p | 

0 s X 55 60 


Figure 4.4 Distributions of Banker A's and Banker B's arrival times 


Probability of triangle 


A stick is cut twice randomly (each cut point follows a uniform distribution on the stick), 
what is the probability that the 3 segments can form a triangle??? 


Solution: Without loss of generality, let's assume that the length of the stick is 1. Let's 
also label the point of the first cut as x and the second cut as y . 


If x € y, then the three segments are x, y-x and 
l-y. The conditions to form a triangle are 


X y-x l-y 
x*(y-x)»1l-y2y»1/2 
x+(l-y)>y-x>y<l/24+x 
(y-x)4*(l—- y)» x 2x «1/2 


The feasible area is shown in Figure 4.5. The E 
case for x € y is the left gray triangle. Using NZ 

symmetry, we can see that the case for x > y is " Ve X L 
the right gray triangle. Figure 4.5 Distribution of cuts X and Y 


? Hint: Let the first cut point be x, the second one be y, use the figure to show the distribution of x and y. 
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The total shadowed area represents the region where 3 segments can form a triangle, 
which is 1/4 of the square. So the probability is 1/4. 


Property of Poisson process 


You are waiting for a bus at a bus station. The buses arrive at the station according to a 
Poisson process with an average arrival time of 10 minutes (A = 0.1/min). If the buses 
have been running for a long time and you arrive at the bus station at a random time, 
what is your expected waiting time? On average, how many minutes ago did the last bus 
leave? 


Solution: Considering the importance of jump-diffusion processes in derivative pricing 
and the role of Poisson processes in studying jump processes, let’s elaborate more on 
exponential random variables and the Poison process. Exponential distribution is widely 
used to model the time interval between independent events that happen at a constant 


Ae“ (t20) 
0 («0) 
and the variance is 1/ A^. Using integration, we can calculate the cdf of an exponential 
distribution to be F(r) 2 P(cxt) 21-e" and P(t >t)=e", where t is the random 
variable for arrival time. One unique property of exponential distribution is 
memorylessness: Pír >s+t|T >s}= P(r»t).^ That means if we have waited for s 


time units, the extra waiting time has the same distribution as the waiting time when we 
start at time 0. 


average rate (arrival rate) A: f(t) -| The expected arrival time is 1/A 


When the arrivals of a series of events each independently follow an exponential 
distribution with arrival rate 4, the number of arrivals between time O and ¢ can be 


ear 
x! 
number of arrivals is At and the variance is also At. Because of the memoryless nature 
of exponential distribution, the number of arrivals between time s and ¢ is also a Poisson 
e ^9 (A-s) 


x! 


modeled as a Poisson process P(N(t)=x)= , x=0,1,---™ The expected 


process P(N(t—s) 2 x)= 


Taking advantage of the memoryless property of exponential distribution, we know that 
the expected waiting time is 1/ 4 2 10min. If you look back in time, the memoryless 


property stills applies. So on average, the last bus arrived 10 minutes ago as well. 


3 P{r>stt|e>sp=e*OM le* =e" —P(x»t) 
More rigorously, M(t) is defined as a right-continuous function. 
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This is another example that your intuition may misguide you. You may be wondering 
that if the last bus on average arrived 10 minutes ago and the next bus on average will 
arrive 10 minutes later, shouldn’t the average arrival time be 20 minutes instead of 10? 
The explanation to the apparent discrepancy is that when you arrive at a random time, 
you are more likely to arrive in a long time interval between two bus arrivals than in a 
short one. For example, if one interval between two bus arrivals is 30 minutes and 
another is 5 minutes, you are more likely to arrive at a time during that 30-minute 
interval rather than 5-minute interval. In fact, if you arrive at a random time, the 


E[X?] 
2E[X] 


expected residual life (the time for the next bus to arrive) is for a general 


distribution.” 


Moments of normal distribution 


If X follows standard normal distribution ( X ~ N(0, 1) ), what is E[X"] forn = 1, 2, 3 
and 4? 


Solution: The first to fourth moments of the standard normal distribution are essentially 
the mean, the variance, the skewness and the kurtosis. So you probably have 
remembered that the answers are 0, 1, 0 (no skewness), and 3, respectively. 


Standard normal distribution has pdf f(x)- gU Using simple symmetry we 
J2z 


have E[x"]- f m~ ^dx =0 when n is odd. For n = 2, integration by parts are 
a 


often used. To solve E[X"] for any integer n, an approach using moment generating 
functions may be a better choice. Moment generating functions are defined as 


^e p(x), if x is discrete 
M(t)= Ele” ]=3 * 
[e f(x)dx, if x is continuous 


Sequentially taking derivative of M(t), we get one frequently-used property of M(t): 
d tX tX t 

M (t) = ]-£[Xe" ] > M (0) = E[X], 

M"(t)= Z E[Xe" | = E[X "e" ]= M'(0) = E[X], 


^ The residual life is explained in Chapter 3 of “Discrete Stochastic Process" by Robert G. Gallager. 
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and M"(0)- ELX"], Vn 21 in general. 


We can use this property to solve E[LX"] for X —N(0,1). For standard normal 


l i1 2 l EOS 2 
e" "dioe p eZ dy pe 
V2 N 


distribution M(t) = E[e" ]= fe Iq 


l 2 ; 
(—=e 1”? is the pdf of normal distribution X ~ N(t, 1), so x)dx =1). 
Jos p [ J 
Taking derivatives, we have 
M (t) =t"  M'(0)20, M(H) =e"? ee ^ => M"(0) =e =1, 
M? (t) 2 te" + 2te" P + Pe"? = 3te" +e" = M? 0) =0, 


and M*(t) 23e? «3e "? +307e" ? care? 5 M*(0) =3e° =3. 


4.5 Expected Value, Variance & Covariance 


Expected value, variance and covariance are indispensable in estimating returns and 
risks of any investments. Naturally, they are a popular test subject in interviews as well. 
The basic knowledge includes the following: 


If E[x,] is finite for all i=1,---,n, then ELX,+---+X,]=ELX,]+---+ E[ X,]. The 
relationship holds whether the x, ’s are independent of each other or not. 
If X and Y are independent, then E[g( X)A(Y)] = Elg(x)JE[A(Y)] . 
Covariance: Cov( X,Y) = E[(X — ELX (Y - EYD] = ELXY]- ELXJE[Y]. 
Cov( X,Y) 
War(X)Var(¥) 


If X and Y are independent, Cov(X,Y)=0 and p(X,Y)-0. EM 


Correlation: p(X,Y) = 


General rules of variance and covariance: 


Co Ya, X,» 324 = S Y ab Cov X,, Yi) 
i=l j=l 


i=) j=l 


Var(Y- X,)= Y Var(X,) +25.9, Cov(X,, X;) 


i<j 
?6 The reverse is not true. p(.X,Y) =0 only means X and Y are uncorrelated; they may well be dependent. 
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Conditional expectation and variance 


For discrete distribution: E[g(X)|Y = y]= > 2(X) Pyy (x ly)= > (x) A(X =x|Y=y) 


For continuous distribution: E[g( X)| Y 2 y] = [ 2(X) f y (x | y)dx 


Law of total expectation: 


> E[X | Y =y]p(Y = y), for discrete Y 
E[X]- ELE[X | Y] 24 2 
[ E[X | Y = y]f, (y)dy, for continuous Y 


Connecting noodles 


You have 100 noodles in your soup bowl. Being blindfolded, you are told to take two 
ends of some noodles (each end on any noodle has the same probability of being chosen) 
in your bowl and connect them. You continue until there are no free ends. The number 
of loops formed by the noodles this way is stochastic. Calculate the expected number of 
circles. 


Solution: Again do not be frightened by the large number 100. If you have no clue how 
to start, let's begin with the simplest case where n =1. Surely you have only one choice 
(to connect both ends of the noodle), so E[ f(1)]=1. How about 2 noodles? Now you 


4x3 _ 
2 


combinations. Among them, 2 combinations will connect both ends of the same noodle 
together and yield 1 circle and 1 noodle. The other 4 choices will yield a single noodle. 
So the expected number of circles is 


6 


4 
have 4 ends (2 x2 ) and you can connect any two of them. There are Bh! > 


ELf (2)]=2/6x (1+ ELf()]) - 4/6x EL f()] =1/3+ ELA (0] 21/341. 


. 6 ; ; 
We now move on to 3 noodles with B = : Z= 15 choices. Among them, 3 choices 


will yield 1 circle and 2 noodles; the other 12 choices will yield 2 noodles only, so 
E[ f(3)] =3/15 x (1+ EL£(2)) - 12/15x E[f(2)]=1/5+ E[ f(2)] = 1/5 - 1/34 1. 


See the pattern? For any n noodles, we will have E[ f(m)]=1+1/34+1/5+---+1/(2n-)), 
which can be easily proved by induction. Plug 100 in, we will have the answer. 
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Actually after the 2-noodle case, you probably have found the key to this question. If 


2 s 
you start with » noodles, among S = n(2n — 1) possible combinations, we have 


"EN probability to yield 1 circle and n—1 noodles and ini probability 
n(2n-1) 2n-1 2n —-] 


to yield » -1 noodles only, so E[ f(n)|= ELf (n —1)] 


. Working backward, you 
2n-] 


can get the final solution as well. 


Optimal hedge ratio 


You just bought one share of stock A and want to hedge it by shorting stock B. How 
many shares of B should you short to minimize the variance of the hedged position? 


Assume that the variance of stock A’s return is o7; the variance of B's return is 0%; 


their correlation coefficient is p. 


Solution: Suppose that we short h shares of B, the variance of the portfolio return is 
var(r, —hr,) = o; -2pho ,0; + h^ o; 


The best hedge ratio should minimize var(r,—Ar,). Take the first order partial 
Ó var 


--2po,0,42ho]-0— h=p—4. 
O5 


derivative with respect to h and set it to zero: 


To confirm it's the minimum, we can also check the second-order partial derivative: 


0^ var 3 c, - 
Ay? — 20; » 0. So Indeed when a the hedge portfolio has the minimum 


B 


variance. 


Dice game 


Suppose that you roll a dice. For each roll, you are paid the face value. Ifa roll gives 4, 5 
or 6, you can roll the dice again. Once you get 1, 2 or 3, the game stops. What is the 
expected payoff of this game? 


Solution: This is an example of the law of total expectation. Clearly your payoff will be 
different depending on the outcome of first roll. Let E[.X] be your expected payoff and 


Y be the outcome of your first throw. You have 1/2 chance to get Y e {1,2,3}, in which 
case the expected value is the expected face value 2, so ELX |Y e {1,2,3}]=2; you have 
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1/2 chance to get Y e (4,5, 6}, in which case you get expected face value 5 and extra 


throw(s). The extra throw(s) essentially means you start the game again and have an 
extra expected value E[X]. So we have ELX |Y €(4,5,6)]] 2 5-- E[.X]. Apply the law of 


total expectation, we have ELX]= E[E[X |Y]]- 2x2 1x(5- E[.X]) > E[X] 7^ 


Card game 


What is the expected number of cards that need to be turned over in a regular 52-card 
deck in order to see the first ace? 


Solution: There are 4 aces and 48 other cards. Let's label them as card 1,2,---,48. Let 


i 


E h if card i is turned over before 4 aces 


0, otherwise 
The total number of cards that need to be turned over in order to see the first ace is 


48 48 
X =1+¢ X ,, So we have E[X J=1+) E[X ,]. As shown in the following sequence, 


i=] i=] 


each card i is equally likely to be in one of the five regions separated by 4 aces: 
1A2A3A4A5 


So the probability that card i appears before all 4 aces is 1/5, and we have ELX,]=1/5. 


48 
Therefore, E[X]=1+ 5 E[X,]=1+48/5=10.6. 
i=] 
This is just a special case for random ordering of m ordinary cards and n special cards. 
The expected position of the first special card is 1+ > ELX J=1+ ee ; 
n+ 


i=] 


Sum of random variables 


Assume that X,, X,,°::, and X, are independent and identically-distributed (IID) 


random variables with uniform distribution between 0 and 1. What is the probability 
that S, = X, +X, +-+ X, <1?” 


*7 You will also see that the problem can be solved using Wald’s equality in Chapter 5. 
^ Hint: start with the simplest case where n =1, 2, and 3. Try to find a general formula and prove it using 
induction. 
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Solution: This problem is a rather difficult one. The general principle to start with the 
simplest cases and try to find a pattern will again help you approach the problem; even 
though it may not give you the final answer. When n=1, P(S, <1) is 1. As shown in 
Figure 4.6, when n=2, the probability that X,+X,<1 is just the area under 
X, +X, <1 within the square with side length 1 (a triangle). So P(S, <1) 21/2. When 
n = 3, the probability becomes the tetrahedron ABCD under the plane X, + X, +X, <1 
within the cube with side length 1. The volume of tetrahedron ABCD is 1/6. ? So 
P(S, <1)=1/6. Now we can guess that the solution is 1/!. To prove it, let's again 
resort to induction. Assume P(S,<1)=1/n!. We need to prove that 
P(S,,, Sl) l/(n4 1)!. 


n 


i 
L] 
ï 
L] 
L] 
i 
L] 
L] 
LI 
a |! 
ï 


* 
"t 


0 X, l 
n=2 
Figure 4.6 Probability that S, < 1 when n = 2 or n=3. 


Here we can use probability by conditioning. Condition on the value of X,,,, we have 


POS, 1)» h AX )P(S, &1— X, )aX 
function of X,,,, so f/(X,,,) —1. But how do we calculate P(S, <1- X)? The cases 


„ Instead of S, <1, 
we essentially need to shrink every dimension of the n-dimensional simplex? from | to 


n+l? 


where f(X,,,) is the probability density 


nel? 


of n-2 and n=3 have provided us with some clue. For S, €1— X 


fi 


? You can derive it by integration: [ A(z)dz = [1/227 =1/6, where A(z) is the cross-sectional area. 


? An n-Simplex is the n-dimensional analog of a triangle. 
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1- X 


n+1° 


| 
(i= Sa ax HS IXY i Edi l l 
we have P(S, <1)= [= Xa = m Eva (n1)! 


So its volume should be 


=A au) 
n' 


instead of ES Plugging in these results, 
n! 


So the general result is true for n+1 as well and we have P(S, <1)=1/n!. 


Coupon collection 


There are N distinct types of coupons in cereal boxes and each type, independent of prior 
selections, is equally likely to be in a box. 


A. If a child wants to collect a complete set of coupons with at least one of each type, 
how many coupons (boxes) on average are needed to make such a complete set? 


B. If the child has collected n coupons, what is the expected number of distinct coupon 


types?’ : 


Solution: For part A, let X,, i=1,2,---, N , be the number of additional coupons needed 
to obtain the i-th type after (;—1) distinct types have been collected. So the total number 
N 
of coupons needed is X = X,+ X,+---+Xy =) X. 
i=] 
For any i, i—1 distinct types of coupons have already been collected. It follows that a 
new coupon will be of a different type with probability 1—(;-1)/ N =(N-i+1)/N. 
Essentially to obtain the i-th distinct type, the random variable X, follows a geometric 
distribution with p -(N —i-1)/ N and E[X,]2 N/(N —i+1). For example, if ; 21, we 
simply have X, = ELX,]=1. 
S~ N l l l 
~” E(X E| X, = N| —+ +e). 
- ax Y Ex Yu x x ] 


i41 


?! Hint: For part A, let X ; be the number of extra coupons collected to get the i-th distinct coupon after 


i—1 types of distinct coupons have been collected. Then the total expected number of coupons to collect 


N 

all distinct types is E[ Y]— > E[X,]. For part B, which is the expected probability (P) that the i-th 
i=] 

coupon type is not in the » coupons? 
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For part B, let Y be the number of distinct types of coupons in the set of » coupons. We 
introduce indicator random variables /,,i — 1,2, ---, N , where 


‘ , =1, if at least one coupon of the i-th type is in the set of n coupons 


I, =0, otherwise 


N 
So we have Y=/,4+/,+---+1, = 


i=l 


For each collected coupon, the probability that it is not the i-th coupon type is -— 3 


Since all n coupons are independent, the probability that none of the n coupons is the i-th 


coupon type is P(/, 20) = e and we have E[/,] - PU, 1) «1 (I ; 


-. E[Y]= * E[,]- N- (=*=) P" 


iz] 


Joint default probability 


If there is a 50% probability that bond A will default next year and a 30% probability 
that bond B will default. What is the range of probability that at least one bond defaults 
and what is the range of their correlation? 


Solution: The range of probability that at least one bond defaults is easy to find. To have 
the largest probability, we can assume whenever A defaults, B does not default; 
whenever B defaults, 4 does not default. So the maximum probability that at least one 
bond defaults is 50% +30% = 80%. (The result only applies if P(A)+ P(B)X1). For 
the minimum, we can assume whenever 4 defaults, B also defaults. So the minimum 
probability that at least one bond defaults is 50%. 

To calculate the corresponding correlation, let /, and /, be the indicator for the event 
that bond A/B defaults next year and p,, be their correlation. Then we have 


E[1,]=0.5, E[/,] 2 0.3, var(1,) = p,x(1— p,) = 0.25, var(/,) 20.21. 


32 A similar question: if you randomly put 18 balls into 10 boxes, what is the expected number of empty 
boxes? 
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P(A or B defaults) = E[I,]+ E[7,]— El LJ] 

= ELI,]- Ell, ] - (EU ,]ELU S] - cov ,.1,)) 

= 0.5+0.3-(0.5x0.3-p,,0 0; ) 

-0.65-40.21/2p,, 
For the maximum probability, we have 0.65 =a 0201/2 Pag =0.8 > Pap = -31/7 . 
For the minimum probability, we have 0.65- V0.21/2 Pag =0.5 > Pi = VTA 
In this problem, do not start with P(A or B defaults)= 0.65 —4/0.21/2 p,, and try to set 


Pug =+1 to calculate the maximum and minimum probability since the correlation 


cannot be +1. The range of correlation is restricted to | -V3/ 7, V3/ 7]. 


4.6 Order Statistics 


Let X be a random variable with cumulative distribution function F,(x). We can derive 
the distribution function for the minimum Y, = min( X,, X,,---, X, ) and for the maximum 
Z, = max(X,, X,,---, X,) ofn IID random variables with cdf F, (x) as 

P(Y, 2 x) = (P(X 2 x)" 21- F, (x) = 0 - F,G))' => fy 2) = fy G)Y0- FG) 
P(Z,Sx)-(P(X€x)) > Fz (x) = (Fy (x) > Jz, (x) = nf (x)(Fy (x))"" 


Expected value of max and min 


Let X,,X,,°::,X, be IID random variables with uniform distribution between 0 and 1. 


What are the cumulative distribution function, the probability density function and 
expected value of Z, =max(X,,X,,--:,X,)? What are the cumulative distribution 


function, the probability density function and expected value of Y, = min(X,, X, ,---, X,)? 


Solution: This is a direct test of textbook knowledge. For uniform distribution on [0,1], 
F,(x)-x and f,(x)=1. Applying F,(x) and f,(x)to Z, ^ max(X,, X,,---, X,) we 
have 


P(Z, < x) = (P(X < x) > F, (2) = (FQ =x" 
—? Jz, (x) = nf , (x) CF, (x) = nx" 
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and E[Z,]- IEA (x)dx = [ nx" dx I7], M 


Applying F,(x) and f,(x)to Y, = min(X,, X,,--., X,) we have 


PUY, 2x) = (P(X > x)" > F, (x) -1-0- F, œ) =1-(1- x)" 
=> fy (x) = nf. G0 Fy x)" = n0-3)" 


and E[Y, |= [ma-3"&- [nü-»»"&-[vT im - — 


Correlation of max and min 


Let X, and X, be IID random variables with uniform distribution between 0 and 1, 
Y=min(X,,X,) and Z=max(X,,X,). What is the probability of Y 2 y given that 


Z <z forany y,z e[0, 1]? What is the correlation of Y and Z? 


Solution: This problem is another demonstration that a figure is worth a thousand words. 
As shown in Figure 4.7, the probability that Z € z is simply the square with side length 


z. So P(Z€z)-z'. Since Z=max(X,,X,) and Y=min(X,,X,), we must have 
Y € Z for any pair of X, and X,. So if y>z, P(Y2y|Z<z)=0. For y €z, that X, 
and X, satisfies Y > y and Z <z is the square with vertices (y, y), (z, y),(z, z), and 


(y, z), which has an area (z— y). So P(Y => yre^Z€z) 2 (z - y) . Hence 


2 2 e 
pureyizsay=| 07? /z^, if 0<z<land0<y<z | 
0, otherwise 


Now let's move on to calculate the correlation of Y and Z. 


E 2872 ead NR E[YZ] - E[Y]E[Z] 


std(Y)xstd(Z) — J EY] ELY x J E|Z?]- E[ZT 
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X? 
l 
Z 
y r2 : = - — —y i = 

pure ode Ls eau ers 

[- sae ae | 

pose. EE "— P 

y Z ] x, 


Figure 4.7 Distribution of X4, Xz, their maximum and minimum. 


Using previous problem’s conclusions, we have E[Y] ——-— 3° E[Z EET 
From the pdfs of Y and Z, f, (x)  n(1— x)" - 2(1— x) and PAR 7- = 2z, we can 
also get E[Y?] - [2a- - yy ‘dy = 2-2 =— d E[Z?] - [224 ==, which give us the 


variances: var(Y) = EDY*] -ATY T zz =- and var(z)=2 (=) DÀ 


To calculate E[YZ], we can use E[YZ] = f [ yzf (y, z)dydz. To solve this equation, we 
need f(y,z). Let's again go back to Figure 4.7. From the figure we can see that when 
0<z<l and O0< y<z, F(y,z) is the shadowed area with probability 


F(y,z)=P(Y x yo Z &z)- P(Z&Ez)- P(Y 2 ycZ &z) z! -(z—- yy 22zy- y' 
"WE. E EN _ 2 E35. 1. 
$2) = FF.) =2 and E[YZ]= [ [ 2yzdydz = [ zb» kaze |z d=- 

3 You may have noticed that var(Y) = var(Z) and wonder whether it is a coincidence for n= 2. It is 
actually true for all integer n. You may want to think about why that is true without resorting to 


calculation. Hint: var(x) = var(1— x) for any random variable x. 
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An alternative and simpler approach to calculate E[YZ] is again to take advantage of 
symmetry. Notice that no matter x, €x, or x,>x,, we always have yz= xx, 


(z = max(x,,x;) and y 2 min(x,, x,)). 
Z 2 ape 
<. EYZ] = Í [ odd = E[X|]E[ X; ] - ) 


cov(Y,Z) E 
Jvar(Y) xA var(Z) 2 


Sanity check: That Y and Z have positive autocorrelation make sense since when Y 
becomes large, Z tends to become large as well (Z 2 Y ). 


Hence cov(Y,Z) = E[YZ]- E[Y]E[Z]= P and corr(Y,Z) = 


Random ants 


500 ants are randomly put on a 1-foot string (independent uniform distribution for each 
ant between 0 and 1). Each ant randomly moves toward one end of the string (equal 
probability to the left or right) at constant speed of 1 foot/minute until it falls off at one 
end of the string. Also assume that the size of the ant is infinitely small. When two ants 
collide head-on, they both immediately change directions and keep on moving at 1 
foot/min. What is the expected time for all ants to fall off the string?" 


Solution: This problem is often perceived to be a difficult one. The following 
components contribute to the complexity of the problem: The ants are randomly located; 
each ant can go either direction; an ant needs to change direction when it meets another 
ant. To solve the problem, let’s tackle these components. 


When two ants collide head-on, both immediately change directions. What does it mean? 
The following diagram illustrates the key point: 


Before collision: ——> «—L— ; After collision: &—— —* > ; switch label: «4—— —4— 


When an ant A collides with another ant B, both switch direction. But if we exchange the 
ants’ labels, it's like that the collision never happens. A continues to move to the right 
and B moves to the left. Since the labels are randomly assigned anyway, collisions make 
no difference to the result. So we can assume that when two ants meet, each just keeps 
on going in its original direction. What about the random direction that each ant chooses? 
Once the collision is removed, we can use symmetry to argue that it makes no difference 
which direction that an ant goes either. That means if an ant is put at the x-th foot, the 


* Hint: If we switch the label of two ants that collide with each other, it's like that the collision never 
happened. 
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expected value for it to fall off is just x min. If it goes in the other direction, simply set x 
to 1— x. So the original problem is equivalent to the following: 


What is the expected value of the maximum of 500 IID random variables with uniform 
distribution between 0 and 1? 


Clearly the answer is s min, which is the expected time for all ants to fall off the 


string. 
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Chapter 5 Stochastic Process and Stochastic Calculus 


In this chapter, we cover a few topics—Markov chain, random walk and martingale, 
dynamic programming—that are often not included in introductory probability courses. 
Unlike basic probability theory, these tools may not be considered to be standard 
requirements for quantitative researchers/analysts. But a good understanding of these 
topics can simplify your answers to many interview problems and give you an edge in 
the interview process. Besides, once you learn the basics, you'll find many interview 
problems turning into fun-to-solve math puzzles. 


5.1 Markov Chain 


A Markov chain is a sequence of random variables X,, X,,---, X,,--- with the Markov 


property that given the present state, the future states and the past states are independent: 
P(X,;7J|X,7i, X, Hh ^; Xo 7) p; = PIX, =I |X, = i) for all p i, ---, 


i i, and j, where i, j € (1, 2, ---, M} represent the state space S —(5,, s,, ..., 5,,} of 


n-1? 


X. 


In other words, once the current state is known, past history has no bearing on the future. 
For a homogenous Markov chain, the transition probability from state i to state j does 
not depend on n.' A Markov chain with M states can be completely described by an 
MxM transition matrix P and the initial probabilities P(.X,) . 


Py P2 ^" Pim 
Transition matrix: Pay p,|- Pa Pa u Pam , where p, is the transition 
Pm Pum `“ Pum 


probability from state i to state jJ. 
M 
Initial probabilities: P(X,) = (P(X, =1), P(X, 22), ++, P(X, = M)), X P(X, =i)=1. 
i=] 
The probability of a path: P(X, =}, X, =i, ---,X, =i,| Xo =i) = Pi Pi, Pii 
Transition graph: A transition graph is often used to express the transition matrix 


graphically. The transition graph is more intuitive than the matrix, and it emphasizes 


' In this chapter, we only consider finite-state homogenous Markov chains (i.e., transition probabilities do 
not change over time). 


Stochastic Process and Stochastic Calculus 


possible and impossible transitions. Figure 5.1 shows the transition graph and the 
transition matrix of a Markov chain with four states: 


1 

T i= 1 2 3 4 + 

0 05 0 05]! 

0.5 i SS pe 05 0 0.25 0.25] 2 
0 04 04 0213 

= ODL. 0 0 0 1 | 4 


0.4 


Figure 5.1 Transition graph and transition matrix of the Play 


Classification of states 
State j is accessible from state i if there is a directed path in the transition graph from i to 
j (3n such that p" 20) Let T, = min(n: X, = J | X, =i), then P(T, «o» 0 ) if and 


only if state j is accessible from state i. States i and j communicate if i is accessible 
from j and j is accessible from ;. In Figure 5.1, state 3 and 1 communicate. State 4 is 
accessible form state 1, but they do not communicate since state 1 is not accessible from 
state 4. 


We say that state ; is recurrent if for every state j that is accessible from i, i is also 
accessible from j (Vj, P(T, «oo) > 0— P(T, «o0) 2 1). A state is called transient if it is 


not recurrent ( Jj, P(T, «o)» 0 and P(T, «o») «1). In Figure 5.1, only state 4 is 


recurrent. States 1, 2 and 3 are all transient since 4 is accessible from 1/2/3, but 1/2/3 are 
not accessible from 4. 


Absorbing Markov chains: A state i is called absorbing if it is impossible to leave this 
state ( p, =1, p; 2 0, Vj €i). A Markov chain is absorbing if it has at least one absorbing 


state and if from every state it is possible to go to an absorbing state. In Figure 5.1, state 
4 is an absorbing state. The corresponding Markov chain is an absorbing Markov chain. 


Equations for absorption probability: The probability to reach a specific absorbing 
state $s, a,,-:-,a,,, are unique solutions to equations a, -l, a,=0 for all absorbing 


M 
state(s) i5, and a, -»a ;D; for all transient states i. These equations can be easily 
j- 
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derived using the law of total probability by conditioning the absorption probabilities on 
the next state. 


Equations for the expected time to absorption: The expected times to absorption, 
A», Hy, are unique solutions to the equations x, = 0 for all absorbing state(s) i and 


H; 71-4 25 p,H, for all transient states i. These equations can be easily derived using the 
j=l 

law of total expectation by conditioning the expected times to absorption on the next 

state. The number 1 is added since it takes one step to reach the next state. 


Gambler’s ruin problem 


Player M has $1 and player N has $2. Each game gives the winner $1 from the other. As 
a better player, M wins 2/3 of the games. They play until one of them is bankrupt. What 
is the probability that M wins? 


Solution: The most difficult part of Markov chain problems often lies in how to choose 
the right state space and define the transition probabilities P, ’s, Vi, j. This problem has 


fairly straightforward states. You can define the state space as the combination of the 
money that player M has ($m) and the money that player N has ($n): 
{(m,n)} = {(3, 0), (2, 1), (01, 2),(0,3)}. (Neither m nor n can be negative since the whole 
game stops when one of them goes bankrupt.) Since the sum of the dollars of both 
players is always $3, we can actually simplify the state space using only m: 
{m} = {0,1, 2,3}. 


The transition graph and the corresponding transition matrix are shown in Figure 5.2. 


Por Po» Pos 1 0 0 0 

P= P, T Da Piz Pis " 4^ 027540 

x 137 20 PD» P22 P» 0 4 0 Z 
20 P» P22 P» 0 0 0 1] 


Figure 5.2 Transition matrix and transition graph for Gambler's ruin problem 


The initial state is X, =1 (M has $1 at the beginning). At state 1, the next state is 0 (M 


loses a game) with probability 1/3 and 2 (M wins a game) with probability 2/3. So 
Pio 71/3 and p,, 2 2/3. Similarly we can get p,, =1/3 and p,, - 2/3. Both state 3 


(M wins the whole game) and state O (M loses the whole game) are absorbing states. 


To calculate the probability that M reaches absorbing state 3, we can apply absorption 
probability equations: 
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3 3 
a, =l,a,=0, and a, =) pda a, - pd. 
j=0 j-0 


Plugging in the transition probabilities using either the transition graph or transition 
a, rae, fa =4/7 
=> 


matrix, we have 
a, -1/3xa, * 2/3x1 a, =6/7 


So, starting from $1, player M has 4/7 probability of winning. 


Dice question 


Two players bet on roll(s) of the total of two standard six-face dice. Player A bets that a 
sum of 12 will occur first. Player B bets that two consecutive 7s will occur first. The 
players keep rolling the dice and record the sums until one player wins. What is the 
probability that A will win? 


Solution: Many of the simple Markov chain problems can be solved using pure 
conditional probability argument. It is not surprising considering that Markov chain is 
defined as conditional probability: 


PiX = j| X, i, X, i s Xo = ho} = p, = PIX 


n+l n+l 


= J|X, =. 

So let’s first solve the problem using conditional probability arguments. Let P(A) be the 
probability that 4 wins. Conditioning P(A) on the first throw’s sum F, which has three 
possible outcomes F =12, F =7 and F ¢ {7,12}, we have 

P(A) = P(A| F =12)P(F =12)+ P(A| F =7)P(F =7)+ P(A F € {7,123) PCF € (7,12]) 
Then we tackle each component on the right hand side. Using simple permutation, we 
can easily see that P(F 212) 21/36, P(F =7)=6/36, P(F ¢ {7,12}) = 29/36. Also it 
is obvious that P(A| F =12)=1 and P(A| F ¢ {7,12})= P(A). (The game essentially 
starts over again.) To calculate P(A|F =7), we need to further condition on the second 
throw’s total, which again has three possible outcomes: E =12, E=7, and E ¢ {7,12}. 


P(A| F =7)= P(A| F =7,E =12)P(E =12| F =7)+P(A| F =7,E=7)P(E =7|F =7) 
+ P(A| F=7,E ¢ {7,12})P(E € {7,12}| F =7) 
= P(A| F =7, E =12)x1/36+ P(A| F =7,E =7)x6/36 
+ P(A| F=7,E € {7,12})x29/36 
=1x1/364+0x6/36 + P(A)x 29/36 21/364 29/ 36P(A) 


Here the second equation relies on the independence between the second and the first 
rolls. If F=7 and E-12, A wins; if F-7 and E=7, A loses; if F=7 and 
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E ¢ {7,12}, the game essentially starts over again. Now we have all the necessarily 
information for P(A). Plugging it into the original equation, we have 


P(A) = P(A| F =12)P(F =12)+ P(A| F =7)P(F =7)+ P(A| F ¢{7,12}) PCF € {7,123) 
=1x1/36 +6/36x (1/36 + 29/36P(A)) +29/36P(A) 


Solving the equation, we get P(A) 2 7/13. 


This approach, although logically solid, is not intuitively appealing. Now let’s try a 
Markov chain approach. Again the key part is to choose the right state space and define 
the transition probabilities. It is apparent that we have two absorbing states, 12 (A wins) 
and 7-7 (B wins), at least two transient states, S (starting state) and 7 (one 7 occurs, yet 
no 12 or 7-7 occurred). Do we need any other states? Theoretically, you can have other 
states. In fact, you can use all combination of the outcomes of one roll and two 
consecutive rolls as states to construct a transition matrix and you will get the same final 
result. Nevertheless, we want to consolidate as many equivalent states as possible. As 
we just discussed in the conditional probability approach, if no 12 has occurred and the 
most recent roll did not yield 7, we essentially go back to the initial starting state S. So 
all we need are states S, 7, 7-7 and 12. The transition graph and probability to reach state 
12 are shown in Figure 5.3. 


] 
6/36 Probability to absorption state 12 
29/36 


ap =l, a,_, =0 


i aş =1/36x1+6/36xa, +29/36xa; p => aş =7/13 
En 5 0 a, =1/36x1+6/36x0+29/36xa, 
1/36 


Figure 5.3 Transition graph and probability to absorption for dice rolls 


Here the transition probability is again derived from conditional probability arguments. 
Yet the transition graph makes the process crystal clear. 


Coin triplets 


Part A. If you keep on tossing a fair coin, what is the expected number of tosses such 
that you can have HHH (heads heads heads) in a row? What is the expected number of 
tosses to have THH (tails heads heads) in a row? 


Solution: The most difficult part of Markov chain is, again, to choose the right state 
space. For the HHH sequence, the state space is straightforward. We only need four 
states: S (for the starting state when no coin is tossed or whenever a 7 turns up before 
HHH), H, HH, and HHH. The transition graph is 
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1/2 1/2 1/2 
OS WS Cm dum) 
1/2 
At state S, after a coin toss, the state will stay at S when the toss gives a T. If the toss 
gives an H, the state becomes H. At state H, it has 1/2 probability goes back to state S if 


the next toss is 7; otherwise, it goes to state HH. At state HH, it also has 1/2 probability 
goes back to state S if the next toss is 7; otherwise, it reaches the absorbing state HHH. 


l 


m CR = = E 

So we have the following transition probabilities: P, =>, P,-2, Pys =F; 
—1 — l — 0 = 

Pa un T72? Pans p. Pun uuu T29 and Pinn HHH =l. 


We are interested in the expected number of tosses to get HHH, which is the expected 
time to absorption starting from state S. Applying the standard equations for the 
expected time to absorption, we have 


Hs - l3 ug * 3 Hg us =14 
Hy =1+3 Hs +7 Man EN uy =12 
Hyg 7 15s * 3 Hun Hy =8 
Hyny =O Hyny =O 
So from the starting state, the expected number of tosses to get HHH is 14. 


Similarly for expected time to reach THH, we can construct the following transition 
graph and estimate the corresponding expected time to absorption: 


1/2 H; =l+4 U; ++ 4r H; =8 

1/2 PES 1/2 l HU, =1+4 Ur ++ Ury HM, =4 

1/2 Maw Gg Hag, =1+44, +} E Hy E2 
1/2 TH 2 Hr T7 Fran TH 

NY Hry = 9 Hrun = 9 


So from the starting state S, the expected number of tosses to get THH is 8. 


Part B. Keep flipping a fair coin until either HHH or THH occurs in the sequence. What 
is the probability that you get an HHH subsequence before THH?” 


* Hint: This problem does not require the drawing of a Markov chain. Just think about the relationship 
between an HHH pattern and a THH pattern. How can we get an HHH sequence before a THH sequence? 
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Solution: Let’s try a standard Markov chain approach. Again the focus is on choosing 
the right state space. In this case, we begin with starting state S. We only need ordered 
subsequences of either HHH or THH. After one coin is flipped, we have either state T or 
H. After two flips, we have states TH and HH. We do not need TT (which is equivalent 
to T for this problem) or HT (which is also equivalent to T as well). For three coin 
sequences, we only need THH and HHH states, which are both absorbing states. Using 
these states, we can build the following transition graph: 


1/2 
1/2 1/2 1/2 1/2 1/2 1/2 ] 
ems vm 5 Om Oma») 
WS 
1/2 
Figure 5.4 Transition graph of coin tosses to reach HHH or THH 


We want to get the probability to reach absorbing state HHH from the starting state S. 
Applying the equations for absorption probability, we have 


Ann = l> ap, =9 a, =Q, azy =0 
==, Jit l zd. 
as —30, t30y a, —3 
| = l 
ar = Zap tzar. Ay = Zap t 50gg an ZF 
cai Je NER ad 
Or, = 3 0,5 * y Op Ogg = 507 +7 ayy aun 7772 


So the probability that we end up with the HHH pattern is 1/8. 


This problem actually has a special feature that renders the calculation unnecessary. You 
may have noticed that a, =0. Once a tail occurs, we will always get THH before HHH. 


The reason is that the last two coins in THH is HH, which is the first two coins in 
sequence HHH. In fact, the only way that the sequence reaches state HHH before THH 
is that we get three consecutive Hs in the beginning. Otherwise, we always have a T 
before the first HH sequence and always end in THH first. So if we don't start the coin 
flipping sequence with HHH, which has a probability of 1/8, we will always have THH 
before HHH. 


Part C. (Difficult) Let’s add more fun to the triplet game. Instead of fixed triplets for the 
two players, the new game allows both to choose their own triplets. Player 1 chooses a 
triplet first and announces it; then player 2 chooses a different triplet. The players again 
toss the coins until one of the two triplet sequences appears. The player whose chosen 
triplet appears first wins the game. 
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If both player 1 and player 2 are perfectly rational and both want to maximize their 
probability of winning, would you go first (as player 1)? If you go second, what is your 
probability of winning? 


Solution: A common misconception is that there is always a best sequence that beats 
other sequences. This misconception is often founded on a wrong assumption that these 
sequences are transitive: if sequence A has a higher probability occurring before 
sequence B and sequence B has a higher probability occurring before sequence C, then 
sequence A has a higher probability occurring before sequence C. In reality, such 
transitivity does not exist for this game. No matter what sequence player 1 chooses, 
player 2 can always choose another sequence with more than 1/2 probability of winning. 
The key, as we have indicated in Part B, is to choose the last two coins of the sequence 
as the first two coins of player 1’s sequence. We can compile the following table for 
each pair of sequences: 


2's winning 
Probability 


5/12 3/10 
Table 5.1 Player 2's winning probability with different coin sequence pairs 


As shown in Table 5.1 (you can confirm the results yourself), no matter what player 1’s 
choices are, player 2 can always choose a sequence to have better odds of winning. The 
best sequences that player 2 can choose in response to 1’s choices are highlighted in 
bold. In order to maximize his odds of winning, player 1 should choose among HTH, 
HTT, THH and THT. Even in these cases, player 2 has 2/3 probability of winning. 


* This problem is a difficult one. Interested reader may find the following paper helpful: "Waiting Time 
and Expected Waiting Time-Paradoxical Situations" by V. C. Hombas, The American Statistician, Vol. 51, 
No. 2 (May, 1997), pp. 130-133. In this section, we will only discuss the intuition. 


112 


A Practical Guide To Quantitative Finance Interviews 


Color balls 


A box contains n balls of n different colors. Each time, you randomly select a pair of 
balls, repaint the first to match the second, and put the pair back into the box. What is 
the expected number of steps until all balls in the box are of the same color? (Very 
difficult) 


Solution: Let N, be the number of steps needed to make all balls the same color, and let 
F,i=1, 2,---, n, be the event that all balls have color i in the end. Applying the law of 


i 


total expectation, we have 

E[N,] - ELN, | F.]PLF ] ELN, | F,]PLF,]+-++ ELN, | F, PLE, ]. 

Since all the colors are symmetric (i.e., they should have equivalent properties), we have 
P[F]- PLE] - -- - P[F,] -1/n and E[N,]- E[N, | E] E[N, | F,]= ELN, | F,]. That 
means we can assume that all the balls have color 1 in the end and use E[N, | F;] to 


represent F[N, ]. 


So how do we calculate E[N, |F]? Not surprisingly, use a Markov chain. Since we 
only consider event F,, color 1 is different from other colors and colors 2, ---, n 


become equivalent. In other words, any pairs of balls that have no color 1 ball involved 
are equivalent and any pairs with a color 1 ball and a ball of another color are equivalent 
if the order is the same as well. So we only need to use the number of balls that have 
color 1 as the states. Figure 5.5 shows the transition graph. 


(8:23.30 92-9 


Figure 5.5 Transition graph for all n balls to become color 1 


State n is the only absorbing state. Notice that there is no state 0, otherwise it will never 
reach F. In fact, all the transition probability is conditioned on F; as well, which makes 


the transition probability p,,,,|, higher than the unconditional probability p,;,, and 
D;,,|F, lower than p,,,. For example, p,,|F, 20 and p,,=1/n. (Without 


conditioning, each ball is likely to be the second ball, so color 1 has 1/7 probability of 
being the second ball.) Using the conditional transition probability, the problem 
essentially becomes expected time to absorption with system equations: 


E[N, | EF] - 1 E[N, , | F]x B, , | K+, | E]x B, |F + EIN, | F]x B, |F. 


E 
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To calculate £,,|H, let's rewrite the probability as P(x,,,=i-1|x, =i, F), 
Vk =0, 1, ..., to make the derivation step clearer: 
P(x, =i,x,,, =i-1,F) 
P(x, =i, Ej) 
| PF, | x, =i-1 x, =D x P(x,,, =i-1| x, 2 i)x P(x, =D) 
P(F |x, =i)x P(x, =i) 
_ PU | xu m i-DxP(x, =i-1 x, =) 


P(x,, -i-l|x, =i,F,)= 


P(F, | x, =i) 
E. i(n —i) 
H "e n(n —1) _ (n-i)x (i-l) 
7 i/n n(n —1) 


The first equation is simply the definition of conditional probability; the second equation 
is the application of Bayes’ theorem; the third equation applies the Markov property. To 
derive P(F |x, =i) - i/n, we again need to use symmetry. We have shown that if all 


the balls have different colors, then we have P[F,]= P[F;] 2 ---— P[F,]-1/n. What is 
the probability of ending in a given color, labeled as c, if i of the balls are of color c? It 
is simply i/n. To see that, we can label the color of each of the 7 balls of color c as 
C; J 2l::,i (even though they are in fact the same color). Now it's obvious that all 
balls will end with color c; with probability 1/n. The probability for c is the sum of 


probabilities of c,'s, which gives the result i/n. 


Similarly we have P(E|x,,,-i-1)-(i-1)/n. For P(x,,,=i-1|x, =), we use a 
basic counting method. There are n(n —1) possible permutations to choose 2 balls out of 


n balls. In order for one color 1 ball to change color, the second ball must be color 1, 
which has i choices; the first ball needs to be another color, which has (n— i) choices. 


i(n — i) 

n(n-1) 

Applying the same principles, we can get 

(n—i)x2i Panen (n—i)x(i 1) | 
n(n-l) ` n(n — 1) 


Plugging into E[N, | F] and simplifying E[N, | F] as Z,, we have 


So P(x,,, =i-l|x, =) = 


P(x,,, -i|x, ci FE) e ———— 


(n—i)x2ixZ, 2 n(n-l)*- (n- iyi -1)Z,, * (n-i(i-1)Z, , 
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Using these recursive system equations and the boundary condition Z, — 0, we can get 
Z =(n-1)." 


5.2 Martingale and Random walk 


Random walk: The process ($,; n 21) is called a random walk if {X,;i21} are IID 
(identical and independently distributed) random variables and S, = X, ---- X,, where 
n=1,2,--- The term comes from the fact that we can think of S, as the position at time 


n for a walker who makes successive random steps X,, X,,--- 


If X, takes values 1 and -1 with probabilities p and 1 — p respectively, S, is called a 
simple random walk with parameter p. Furthermore, if p — 1, the process S, is a 
symmetric random walk. For symmetric random walk, it's easy to show that 
E[S,]=0 and var(S,) = E[S?]- E[S,  » E[S?] 2 n ? 


Symmetric random walk is the process that is most often tested in quantitative 
interviews. The interview questions on random walk often revolve around finding the 
first n for which S, reaches a defined threshold a, or the probability that S, reaches 


a for any given value of n. 

Martingale: a martingale (Z,;n21) is a stochastic process with the properties that 
E||Z, || « » for all n and E[Z,,,|Z, = Zp, Zp = Z, Z -z]-z,. The property of a 
SZ -z|-z,, which 


means the conditional expected value of future Z, is the current value Z,.° 


martingale can be extended to E[Z,; m» n|Z, -z,,Z, =Z 


n-ó `` 


A symmetric random walk is a martingale. From the definition of the symmetric random 
S, *1 with probability 1/2 

S, —-1 with probability 1/2 ' 
Since EISi, -(n* 1) 2H[CS, - 1) - (S, -1)]- (n1) 2 S2 -n, S? —n is a martingale 


n+l 


walk we have S, -| so E[S,,,|S, 25, , S, =5,]=5,. 


as well. 


* Even this step is not straightforward. You need to plug in the i's and try a few cases starting with 
Z 


n-1? “~n-2? 


i -n-]. The pattern will emerge and you can see that all the terms containing Z -:,Z, cancel out. 


? Induction again can be used for its proof. Var(S,) =Var(Z,) 21. Induction step: If Var(S,) =n, then we 
have Var(S 


nil 
* Do not confuse a martingale process with a Markov process. A martingale does not need to be a Markov 
process; a Markov process does not need to be a martingale process, either. 


) - Var(S, +x) 2 Var(S,) -Var(x, ,) 2 n*1 since x is independent of S. 
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Stopping rule: For an experiment with a set of IID random variables X,,X,,::-, a 
stopping rule for {X,;i 21} is a positive integer-value random variable N (stopping time) 
such that for each n » 1, the event {N € n) is independent of X,,,, X 


n-l?^*^n«2? 


--, Basically it 
says that whether to stop at n depends only on X,,X,,-::,X, (i.e., no look ahead). 


Wald’s Equality: Let N be a stopping rule for IID random variables X,, X., and let 
S, =X,+X,4+:::+X,, then E[S,]- ELX ]|E[N]. 


Since it is an important—yet relatively little known—theorem, let's briefly review its 
proof. Let 7, be the indicator function of the event {N 2 nj. So S, can be written as 


S, => X,I, , where ], -l1if N 2 n and I, -0 if N € n-1. 
n=] 


n n? 


From the definition of stopping rules, we know that 7, is independent of X,, X,.,,--- 
(it only depends on X,, X,,-:-, X, ,). So E[X,1,]- E[|X,]E|1,]- E[| X |E[1,] and 


E[S,]- exu. = $ E[X,1]- » E[X]E[I, = E(x] El = E[X]E[N]. 


A martingale stopped at a stopping time is a martingale. 


Drunk man 


A drunk man is at the 17th meter of a 100-meter-long bridge. He has a 50% probability 
of staggering forward or backward one meter each step. What is the probability that he 
will make it to the end of the bridge (the 100th meter) before the beginning (the Oth 
meter)? What is the expected number of steps he takes to reach either the beginning or 
the end of the bridge? 


Solution: The probability part of the problem—often appearing in different disguises—is 
among the most popular martingale problems asked by quantitative interviewers. 
Interestingly, few people use a clear-cut martingale argument. Most candidates either 
use Markov chain with two absorbing states or treat it as a special version of the 
gambler’s ruin problem with p =1/2. These approaches yield the correct results in the 
end, yet a martingale argument is not only simpler but also illustrates the insight behind 
the problem. 


’ For detailed proof and applications of Wald’s Equality, please refer to the book Discrete Stochastic 
Processes by Robert G. Gallager. 
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Let’s set the current position (the 17th meter) to 0; then the problem becomes a 
symmetric random walk that stops at either 83 or -17. We also know that both S, and 
S^ —n are martingales. Since a martingale stopped at a stopping time is a martingale, 
S, and S} -N (where S, =X,+X,+---+X, with N being the stopping time) are 
martingales as well. Let p, be the probability that it stops at a=83, p, be the 
probability it stops at -Ø =-17 ( p; 21- p, ), and N be the stopping time. Then we 


have 

E[S,]= p, x88- (10- p,)x172 S, =0 p, 70.17 
> 

EIS} - N] = E[p, x83? +(1- p,) x17 ]- E[N] = $2 -020 E[N]-1441 


Hence, the probability that he will make it to the end of the bridge (the 100th meter) 
before reaching the beginning is 0.17, and the expected number of steps he takes to 
reach either the beginning or the end of the bridge is 1441. 


We can easily extend the solution to a general case: a symmetric random walk starting 
from 0 that stops at either a (œ » 0) or -p ( B » 0). The probability that it stops at a 
instead of -p is p, = J (a+ B). The expected stopping time to reach either æ or —£ 


is E[N] = af. 


Dice game 


Suppose that you roll a dice. For each roll, you are paid the face value. If a roll gives 4, 5 
or 6, you can roll the dice again. If you get 1, 2 or 3, the game stops. What is the 
expected payoff of this game? 


Solution: In Chapter 4, we used the law of total expectation to solve the problem. A 
simpler approach—requiring more knowledge—is to apply Wald’s Equality since the 
problem has clear stopping rules. For each roll, the process has 1/2 probability of 
stopping. So the stopping time N follows a geometric distribution with p 21/2 and we 


have E[N]-1/ p 7 2. For each roll, the expected face value is E[X]=7/2. The total 
expected payoff is E[S, |= ELX ]|E[.N] 2 7/2x2 =7. 


Ticket line 


At a theater ticket office, 2n people are waiting to buy tickets. n of them have only $5 
bills and the other » people have only $10 bills. The ticket seller has no change to start 
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with. If each person buys one $5 ticket, what is the probability that all people will be 
able to buy their tickets without having to change positions? 


Solution: This problem is often considered to be a difficult one. Although many can 
correctly formulate the problem, few can solve the problem using the reflection 
principle. This problem is one of the many cases where a broad knowledge makes a 
difference. 


Assign +1 to the n people with $5 bills and -1 to the n people with $10 bills. Consider 

the process as a walk. Let (a,b) represent that after a steps, the walk ends at b. So we 

start at (0,0) and reaches (2n,0) after 2n steps. For these 2” steps, we need to choose n 
2n | 

steps as +1, so there are |- 2n- possible paths. We are interested in the paths that 
n n'n! 

have the property b20, V0«a «2n steps. It's easier to calculate the number of 

complement paths that reach b 2 —1, 30« a « 2n. As shown in Figure 5.6, if we reflect 

the path across the line y =-—1 after a path first reaches -1, for every path that reaches 

(2n,0) at step 2n, we have one corresponding reflected path that reaches (2n,-—2) at 

step 2n. For a path to reach (2z,— 2), there are (n — 1) steps of +1 and (n^ 1) steps of -1. 


2 
So there are ES such paths. The number of paths that have the 
n—lj (n-D'(n41)! 


h— 


2 
property b 2 —1, 30 « a «2n, given that the path reaches (27,0) is also | 4 and the 


number of paths that have the property b 20, V0 « a «2n is 


HS Cart 


Hence, the probability that all people will be able to buy their tickets without having to 
change positions is 1/(n+1). 


* Consider a random walk starting at a, S, =a, and reaching b in n steps: S, =b. Denote N (a,b) as the 
number of possible paths from (0,a) to (n, b) and N^(a,b) as the number possible paths from (0,a) to 
(n,b) that at some step k (k>0,), S,=0; in other words, N' (a,b) are the paths that contain 
(k,0), 30 « k <n. The reflection principle says that if a, b » 0, then N°(a,b) = N,(-a,b). The proof is 


intuitive: for each path (0,a) to (k,0), there is a one-to-one corresponding path from (0,—a) to (k,0). 
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Figure 5.6 Reflected paths: the dashed line is the reflection of the solid line 
after it reaches -1 


Coin sequence 


Assume that you have a fair coin. What is the expected number of coin tosses to get n 
heads in a row? 


Solution: Let E[ f(n)] be the expected number of coin tosses to get n heads in a row. In 


the Markov chain section, we discussed the case where n =3 (to get the pattern HHH). 
For any integer n, we can consider an induction approach. Using the Markov chain 
approach, we can easy get that E[/(1)]|-2, E[/(2)]2 6 and E[/(3)] 214. A natural 


guess for the general formula is that E[ f(n)]=2”"*'—2. As always, let's prove the 
formula using induction. We have shown the formula is true for n =1,2,3. So we only 


need to prove that if E[f(n)]|-2"' -2, El f(n+1)]=2”"" -2. The following diagram 
shows how to prove that the equation holds for E[ f (n *1)]: 


Q- GD = eap 


The state before (n+1) heads in a row (denoted as (n+1)H ) must be n heads in a row 
(denoted as nH ). It takes an expected E[f(n)]=2""'-2 tosses to reach nH. 
Conditioned on state nH, there is a 1/2 probability it will go to (n+1)H (the new toss 
yields H) and the process stops. There is also a 1/2 probability that it will go to the 
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starting state 0 (the new toss yields T) and we need another expected E[ f (n +1)] tosses 
to reach (n+1)H. So we have 


E[f(n+))]= E[F(n)]- 3x14 2x E[f (n4 1)] 
=> E[ f(n+1)] 22x E[F(n)]] - 2 = 2" -2 


General Martingale approach: Let's use HH ---H, to explain a general approach for the 


expected time to get any coin sequence by exploring the stopping times of martingales." 
Imagine a gambler has $1 to bet on a sequence of n heads ( HH --- H, ) in a fair game 
with the following rule: Bets are placed on up to n consecutive games (tosses) and each 
time the gambler bets all his money (unless he goes bankrupt). For example, if H 
appears at the first game, he will have $2 and he will put all $2 into the second game. He 
stops playing either when he loses a game or when he wins n games in a roll, in which 
case he collects $2" (with probability 1/2"). Now let's imagine, instead of one gambler, 
before each toss a new gambler joins the game and bets on the same sequence of n heads 
with a bankroll of $1 as well. After the i-th game, i gamblers have participated in the 
game and the total amount of money they have put in the game should be $i. Since each 
game is fair, the expected value of their total bankroll is $i as well. In other words, if we 
denote x, as the amount of money all the participating gamblers have after the i-th game, 


then (x, —i) is a martingale. 


Now, let's add a stopping rule: the whole game will stop if one of the gamblers becomes 
the first to get n heads in a roll. A martingale stopped at a stopping time is a martingale. 
So we still have E[(x,—i)]=0. If the sequence stops after the i-th toss (i27), the 


(i—n+1)-th player is the (first) player who gets n heads in a roll with payoff 2". So all 
the (—7) players before him went bankrupt; the (i-+ 2) -th player gets (n —1) heads 
in a roll with payoff 2"'; ...; the i-th player gets one head with payoff 2. So the total 
payoff is fixed and x, 22" - 2"! +---+2'=2""'-2. 


Hence, E[(x, -i)] 2 2"! -2- E[ji] 202 E[i] 2"! - 2. 


This approach can be applied to any coin sequences—as well as dice sequences or any 
sequences with arbitrary number of elements. For example, let's consider the sequence 
HHTTHH. We can again use a stopped martingale process for sequence HHTTHH. The 
gamblers join the game one by one before each toss to bet on the same sequence 
HHTTHH until one gambler becomes the first to get the sequence HHTTHH. If the 
sequence stops after the i-th toss, the (;j — 5h gambler gets the HHTTHH with payoff 


? If you prefer more details about the approach, please refer to “A Martingale Approach to the Study of 
Occurrence of Sequence Patterns in Repeated Experiments" by Shuo-Yen Robert Li, The Annals of 
Probability, Vol. 8, No. 6 (Dec., 1980), pp. 1171-1176. 
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2°. All the (;—6) players before him went bankrupt; the (i —4)/h player loses in the 
second toss (HT); the (i—3)th player and the (i — 2)/h player lose in the first toss (T); 


the (; - 1h player gets sequence HH with payoff 2^ and the i-th player gets H with 
payoff 2. 


Hence, E[(x, - i)] 2 2f +2? - 2 - E[i] -0— E[i]- 70. 


5.3 Dynamic Programming 


Dynamic Programming refers to a collection of general methods developed to solve 
sequential, or multi-stage, decision problems.' It is an extremely versatile tool with 
applications in fields such as finance, supply chain management and airline scheduling. 
Although theoretically simple, mastering dynamic programming algorithms requires 
extensive mathematical prerequisites and rigorous logic. As a result, it is often perceived 
to be one of the most difficult graduate level courses. 


Fortunately, the dynamic programming problems you are likely to encounter in 
interviews—although you often may not recognize them as such—are rudimentary 
problems. So in this section we will focus on the basic logic used in dynamic 
programming and apply it to several interview problems. Hopefully the solutions to 
these examples will convey the gist and the power of dynamic programming. 


A discrete-time dynamic programming model includes two inherent components: 
1. The underlying discrete-time dynamic system 


A dynamic programming problem can always be divided into stages with a decision 
required at each stage. Each stage has a number of states associated with it. The decision 
at one stage transforms the current state into a state in the next stage (at some stages and 
states, the decision may be trivial 1f there 1s only one choice). 


Assume that the problem has N +1stages (time periods). Following the convention, we 
label these stages as 0,1, ---, N - 1, N. At any stage k, O< k < N-1, the state transition 
can be expressed as x,,, = f/(x,,u,, w,), where x, is the state of system at stage k;'' u, 


is the decision selected at stage k; w, is a random parameter (also called disturbance). 


!? This section barely scratches the surface of dynamic programming. For up-to-date dynamic 
programming topics, I'd recommend the book Dynamic Programming and Optimal Control by Professor 
Dimitri P. Bertsekas. 


'' In general, X, can incorporate all past relevant information. In our discussion, we only consider the 


present information by assuming Markov property. 
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Basically the state of next stage x,,, is determined as a function of the current state x,, 
current decision u, (the choice we make at stage k from the available options) and the 


random variable w, (the probability distribution of w, often depends on x, and u, ). 


2. A cost (or profit) function that is additive over time. 


Except for the last stage (N), which has a cost/profit g,(x,) depending only on x,, the 


costs at all other stages g,(x,,u,,w,) can depend on x,, 4,, and w,. So the total 
N-I 
cost/profit is gy (xy) * 2g, 04, W) . 
k=i 
The goal of optimization is to select strategies/policies for the decision sequences 
m* = {u,*,-::,Uy_,*} that minimize expected cost (or maximize expected profit): 


N-I 


J (x)= min E {gy (xy) * 5 g, Qu, w,)) 


k=0 


Dynamic programming (DP) algorithm 


The dynamic programming algorithm relies on an idea called the Principle of 
Optimality: If z*= {u,*,---,u, ,*} is the optimal policy for the original dynamic 
programming problem, then the tail policy z,* = {u,*,---,u,_,*} must be optimal for the 


N ] 
tail subproblem E(gy(x,)* 5.8, (x,,U,,W,)}. 


k-i 


N | 
DP algorithm: To solve the basic problem J,.(x,) = min E{g, (x,)+ > & (x,,U,,W,)}, 
E k-0 
start with J, (x, ) - g,(x,), and go backwards minimizing cost-to-go function J,(x,): 
J,G,)- min Eg, (XM MAS Setar Md} oA =0,::-,N—1. Then the J,(x,) 


generated from this algorithm is the expected optimal cost. 


Although the algorithm looks complicated, the intuition is straightforward. For dynamic 
programming problems, we should start with optimal policy for every possible state of 
the final stage (which has the highest amount of information and least amount of 
uncertainty) first and then work backward towards earlier stages by applying the tail 
policies and cost-to-go functions until you reach the initial stage. 


Now let’s use several examples to show how the DP algorithm is applied. 
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Dice game 


You can roll a 6-side dice up to 3 times. After the first or the second roll, if you get a 
number x, you can decide either to get x dollars or to choose to continue rolling. But 
once you decide to continue, you forgo the number you just rolled. If you get to the third 
roll, you'll just get x dollars if the third number is x and the game stops. What is the 
game worth and what is your strategy? 


Solution: This is a simple dynamic programming strategy game. As all dynamic 
programming questions, the key is to start with the final stage and work backwards. For 
this question, it is the stage where you have forgone the first two rolls. It becomes a 
simple dice game with one roll. Face values 1, 2, 3, 4, 5, and 6 each have a 1/6 
probability and your expected payoff is $3.5. 


Now let's go back one step. Imagine that you are at the point after the second roll, for 
which you can choose either to have a third roll with an expected payoff of $3.5 or keep 
the current face value. Surely you will keep the face value if it 1s larger than 3.5; in other 
words, when you get 4, 5 or 6, you stop rolling. When you get 1, 2 or 3, you keep rolling. 
So your expected payoff before the second roll is 3/6x3.5+1/6x(4+5+6) = $4.25. 


Now let's go back one step further. Imagine that you are at the point after the first roll, 
for which you can choose either to have a second roll with expected payoff $4.25 (when 
face value is 1, 2, 3 or 4) or keep the current face value. Surely you will keep the face 
value if it is larger than 4.25; In other words, when you get 5 or 6, you stop rolling. So 
your expected payoff before the first roll is 4/6x 4.25 +1/6x(5+ 6) 2 $14/3. 


This backward approach—called tail policy in dynamic programming gives us the 
strategy and also the expected value of the game at the initial stage, $14/3. 


World series 


The Boston Red Sox and the Colorado Rockies are playing in the World Series finals. In 
case you are not familiar with the World Series, there are a maximum of 7 games and 
the first team that wins 4 games claims the championship. You have $100 dollars to 
place a double-or-nothing bet on the Red Sox. 


Unfortunately, you can only bet on each individual game, not the series as a whole. How 
much should you bet on each game so that if the Red Sox wins the whole series, you win 
exactly $100, and if Red Sox loses, you lose exactly $100? 


Solution: Let (i,j) represents the state that the Red Sox has won i games and the 
Rockies has won j games, and let /(i, j) be our net payoff, which can be negative when 
we lose money, at state (i, j). From the rules of the game, we know that there may be 
between 4 and 7 games in total. We need to decide on a strategy so that whenever the 
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series is over, our final net payoff is either +100—when Red Sox wins the 
championship—or —100—when Red Sox loses. In other words, the state space of the 
final stage includes {(4,0), (4,1), (4,2), (4,3)} with payoff f(i,7)=100 and 
{(0, 4), (1,4), (2,4), (3,4)} with payoff f(i,j)=-—100. As all dynamic programming 
questions, the key is to start with the final stage and work backwards—even though in 
this case the number of stages is not fixed. For each state (i, j), if we bet $y on the Red 
Sox for the next game, we will have (/f(i, 7) + y) if the Red Sox wins and the state goes 
to (+1, j), or (f (4, j) - y) if the Red Sox loses and the state goes to (i, j+1). So 
clearly we have 


f(i+l, j)-fG, i ae D-(fG*l D+ fC, j+1))/2 
fG j*D- f(,J)-» y=(fü+1, j)- fG, j*0)/2 | 


f(4,3)- f(3, 4) 100-100 
2 

with the columns representing ; and the rows representing j. Now we have all the 

information to fill in f(4, 0, f(4, 1), f(4,3, f(42) £(0,4), f(1,4) /fQ,4) 

f 3, 4), as well as f (3,3). Similarly we can also fill in all f'(i, j) for the states where 

i=3 or j 23 as shown in Figure 5.7. Going further backward, we can fill in the net 


For example, we have f(3,3)- — 0. Let's set up a table 


payoffs at every possible state. Using equation y -( fG+l, j)-f(, j+ 1))/ 2, we can 
also calculate the bet we need to place at each state, which is essentially our strategy. 


If you are not accustomed to the table format, Figure 5.8 redraws it as a binomial tree, a 
format you should be familiar with. If you consider that the boundary conditions are 
f(4,0) f(4,1), (4,3), (4,2), /(04) (14), £(2,4), and fC, 4), the 
underlying asset either increases by | or decrease by 1 after each step, and there is no 
interest, then the problem becomes a simple binomial tree problem and the bet we place 
each time is the delta in dynamic hedging. In fact, both European options and American 
options can be solved numerically using dynamic programming approaches. 
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Colorado Rockies 
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Colorado Rockies (| 


100 
(4,0) 
87.5 100 
(3,0) (4,1) 
62.5 75 100 
(2,0) (3,1) (4,2) 
31.25 37.5 50 100 
(1,0) (2,1) (3,2) (4,3) 
0 0 0 0 
(0,0) (1,1) (2,2) (3,3) 
-31.25 -37.5 -50 -100 
(0,1) (1,2) (2,3) (3,4) 
62.5 -75 -100 
(0,2) (1,3) (2,4) 
87.5 100 
(0,3) (1,4) 
-100 
(0,4) 


Figure 5.8 Payoff at different states expressed in a binomial tree 


125 


Stochastic Process and Stochastic Calculus 


Dynamic dice game 


A casino comes up with a fancy dice game. It allows you to roll a dice as many times as 
you want unless a 6 appears. After each roll, if 1 appears, you will win $1; if 2 appears, 
you will win $2; ...; if 5 appears, you win $5; but if 6 appears all the moneys you have 
won in the game is lost and the game stops. After each roll, if the dice number is 1-5, 
you can decide whether to keep the money or keep on rolling. How much are you 
willing to pay to play the game (if you are risk neutral)?" 


Solution: Assuming that we have accumulated n dollars, the decision to have another 
roll or not depends on the expected profit versus expected loss. If we decide to have an 
extra roll, our expected payoff will become 


l l l l l l 5 
—(n+1)+—(n+2)+—(n+3)+—(n+4)+—(n+5)+—x0=—n+2.5. 
A ) «( ) et ) et ) 2 ) ; : 


We have another roll if the expected payoff zn +2.5>n, which means that we should 


keep rolling if the money is no more than $14. Considering that we will stop rolling 
when n 215, the maximum payoff of the game is $19 (the dice rolls a 5 after reaching 


the state n 214 ). We then have the following: /(19)219, f(18)218, /(17)=17, 
f(16)216, and Vi id When n<14, we wil keep on rolling, so 


E[f(n)|nx14]2 — => Al f(n+i)]. Using this equation, we can calculate the value for 


i=l 
EL f (n)] dub for all n 214, 13, ---, 0. The results are summarized in Table 5.2. 


Since E[/(0)] 26.15, we are willing to pay at most $6.15 for the game. 


"E 7 1 15 14 13 12 1 10 


Erfm[19.00 18.00 17.00 16.00 15.00 14.17 13.36 12.59 11.85 11.16 


n b 8 7 6 5 4 3 2 1 0 | 


Etfi) 10.52 9.9] 934 880 829 7.81 736 693 6.53 6415 


Table 5.2 Expected payoff of the game when the player has accumulated n dollars 


les 


Hint: If you decide to have another roll, the expected amount you have after the roll should be higher 
than the amount before the roll. As the number of dollars increases, you risk losing more money if a 6 
appears. So when the amount of dollar reaches a certain number, you should stop rolling. 
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Dynamic card game 


A casino offers yet another card game with the standard 52 cards (26 red, 26 black). The 
cards are thoroughly shuffled and the dealer draws cards one by one. (Drawn cards are 
not returned to the deck.) You can ask the dealer to stop at any time you like. For each 
red card drawn, you win $1; for each black card drawn, you lose $1. What is the optimal 
stopping rule in terms of maximizing expected payoff and how much are you willing to 
pay for this game? 


Solution: It is another problem perceived to be difficult by many interviewees. Yet it is a 
simple dynamic programming problem. Let (b, r) represent the number of black and red 


cards left in the deck, respectively. By symmetry, we have 


red cards drawn — black cards drawn = black cards left — red cards left = b—r 


At each (5, r), we face the decision whether to stop or keep on playing. If we ask the 


dealer to stop at (b, r), the payoff is b—r. If we keep on going, there is i 
tr 


probability that the next card will be black—in which case the state changes to 


(b—1, r) —and — 
b+r 


changes to (b, r - 1). We will stop if and only if the expected payoff of drawing more 
cards is less than b —r. That also gives us the system equation: 


b 
b+r 


probability that the next card will be red—1in which case the state 


r 
b+r 


E| f(b-1,r)]+ 


E| f (b, Je max(5-. [ o) " 


As shown in Figure 5.9 (next page), using the boundary conditions /(0, r)- 0, 
f(b, 0) b, Vb, r=0, 1, ---, 26, and the system equation for E [ f (b, r)]. we can 
recursively calculate E [ f (b, r)| for all pairs of 5 and r. 


The expected payoff at the beginning of the game is E| f(26, 26)] = $2.62. 


^ You probably have recognized this system equation as the one for American options. Essentially you 
decide whether you want to exercise the option at state (b, r). 
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Number of Black Cards Left 


f(b,r) 
0 1 2 3 4 5 6 7 8 9 10 1 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 
0 1 2 3 4 5 6 7 8 9 10 14 #12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 
0 0501 2 3 4 5 6 7 8 9 10 1! #12 13 14 15 16 17 18 19 20 21 22 23 24 25 
o 033 067 1202 3 4 5 6 7 8 9 10 11 12 13 14 15 #16 17 18 19 20 21 22 23 24 
0 025 050 085 1342 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 
0 020 040 066 100 144 2072 3 4 5 6 7 8 9 10 1! 12 13 14 15 16 17 18 19 20 21 22 
O 0.17 033 0.54 0.79 1.12 155 2153 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 zx 
O 0.14 029 045 0.66 091 123 166 2233 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
O 0.13 025 0.39 0.56 076 1.01 134 175 2303 4 5 6 7 8 9 310 11 12 13 #14 15 16 17 418 19 
0 0.11 0.22 0.35 0.49 0.66 0.86 1.11 1.43 1.84 2.36 3.05 4 5 6 7 8 9 10 11 #12 13 14 15 16 17 18 
t O 0.10 0.20 0.31 0.43 0.58 0.75 0.95 1.21 1.52 192 243 3104 5 6 7 8 9 10 11 12 13 144 145 4&6 17 
j O | 0.00 0.18 0.28 0.39 0.52 0.66 0.83 1.04 1.30 161 2.00 250 3154 5 6 7 8 9 10 11 12 13 144 15 16 
2 O 0.08 0.17 0.26 0.35 0.46 0.59 0.74 0.91 1.12 1.38 1.69 2.08 257 3204 5 6 7 8 9 10 11 12 #13 14 15 
= O 0.08 0.15 0.24 0.32 0.42 0.54 0.66 0.81 0.99 1.20 1.46 177 215 263 3244 5 6 7 8 9 10 11 12 #13 «14 
d 0 0.07 0.14 0.22 0.30 0.39 0.49 0.60 0.73 0.89 1.06 1.28 1.53 1.84 222 2.70 3.28 4.03 5 6 7 8 9 10 11 12 43 
9 O 0.07 0.13 0.20 0.28 0.36 0.45 0.55 0.67 0.80 0.95 1.13 1.35 1.60 191 229 275 333 4005 6 7 8 9 10 11 12 
F- O 0.06 0.13 0.19 0.26 0.33 0.42 0.51 0.61 0.73 0.86 1.02 1.20 1.42 1.67 1.98 2.36 2.81 338 4095 6 7 8 9 10 11 
E 0 0.06 0.12 0.18 0.24 0.31 0.39 0.47 0.57 0.67 0.79 0.93 1.08 1.27 1.48 1.74 2.05 242 2.87 3.43 413 5 6 7 8 9 10 
0 0.06 0.11 0.17 0.23 0.29 0.36 0.44 0.53 0.62 0.73 0.85 0.99 1.15 1.33 1.55 1.81 2.11 248 2.93 3.48 4165 6 7 8 9 
0 0.05 0.11 0.16 0.22 0.28 0.34 0.41 0.49 0.58 0.67 0.78 0.90 1.04 1.21 1.39 1.61 1.87 217 2.54 2.99 3.53 4.19 5 6 7 8 
0 0.05 0.10 0.15 0.20 0.26 0.32 0.39 0.46 0.54 0.63 0.73 0.84 0.96 1.10 1.26 1.45 1.67 1.93 2.24 2.60 3.04 3.57 422 5016 7 
O 0.05 0.10 0.14 0.19 0.25 0.31 0.37 0.43 0.51 0.59 0.68 0.78 0.89 1.01 1.16 1.32 1.51 1.73 1.99 2.30 2.66 3.09 3.62 425 5.03 6 
O 0.05 0.09 0.14 0.19 0.24 0.29 0.35 0.41 0.48 0.55 0.63 0.72 0.83 0.94 1.07 1.21 1.38 1.57 1.79 2.05 2.35 2.72 3.15 3.66 4.28 5.05 
O 0.04 0.09 0.13 0.18 0.23 0.28 0.33 0.39 0.45 0.52 0.60 0.68 0.77 0.87 0.99 1.12 1.26 1.43 1.62 1.85 2.11 2.41 2.77 320 3.71 4.32 
0 0.04 0.08 0.13 0.17 0.22 0.26 0.32 0.37 0.43 0.49 0.56 0.64 0.72 0.82 0.92 1.04 1.17 1.32 1.48 1.68 1.90 2.16 2.47 2.82 3.25 3.75 
0 0.04 0.08 0.12 0.16 0.21 0.25 0.30 0.35 0.41 0.47 0.53 0.60 0.68 0.77 0.86 0.97 1.08 1.22 1.37 1.54 1.73 1.96 2.22 2.52 2.88 3.30 
O 0.04 0.08 0.12 0.16 0.20 0.24 0.29 0.34 0.39 0.45 0.51 0.57 0.64 0.72 0.81 0.90 1.01 1.13 1.26 1.42 1.59 1.78 2.01 2.27 2.57 2.93 
O 0.04 0.07 0.11 0.15 0.19 0.23 0.28 0.32 0.37 0.43 0.48 0.54 0.61 0.68 0.76 0.85 0.95 1.06 1.18 1.31 1.46 1.64 1.83 2.06 2.32 2.62 


Figure 5.9 Expected payoffs at different states (b, r) 
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5.4 Brownian Motion and Stochastic Calculus 


In this section, we briefly go over some problems for stochastic calculus, the counterpart 
of stochastic processes in continuous space. Since the basic definitions and theorems of 
Brownian motion and stochastic calculus are directly used as interview problems, we’ll 
simply integrate them into the problems instead of starting with an overview of 
definitions and theorems. 


Brownian motion 
A. Define and enumerate some properties of a Brownian motion?! 


Solution: This is the most basic Brownian motion question. Interestingly, part of the 
definition, such as W(0)=0, and some properties are so obvious that we often fail to 


recite all the details. 
A continuous stochastic process W(t), t 20, is a Brownian motion if 
e W(0)=0; 
e The increments of the process W(t,)-W(0), W(t,) -W (t), ---, W(t.) -W (t), 
VO<¢t, €t, €--- €t, are independent; 
e Each of these increments is normally distributed with distribution 
W (ta) -W (t) ~ NO, t, —t;). 
Some of the important properties of Brownian motion are the following: continuous (no 
jumps); E [WA] = 0; E [We | =f, W(t)~ N(0,t); martingale property 
E|W (t - s)|W()| 2W(t); cov(W(s),W(t))=s, VO<s<t; and Markov property (in 
continuous space). 


There are two other important martingales related to Brownian motion that are valuable 
tools in many applications. 


e Y(t)=W(t) —t is a martingale. 


e Z(t)-expiAW(t) -1LA't$, where 4 is any constant and W(t) is a Brownian 
2 


motion, is a martingale. (Exponential martingale). 


! A Brownian motion is often denoted as B,. Alternatively it is denoted as W(t) since it is a Wiener 


process. In this section, we use both notations interchangeably so that you get familiar with both. 
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We'll show a proof of the first martingale using Ito’s lemma in the next section. A 
sketch for the exponential martingale is the following: 


E[Z(t+s)]=E| exp{A(W@+W(s))-3.27(¢+5)} | 
= exp {AW (t) — ŻA *r} exp|- 14's ! E| exp AW (s)} | 
=Z, exp {-1/?s| exp {tas =Z 


B. What is the correlation of a Brownian motion and its square? 


Solution: The solution to this problem is surprisingly simple. At time ¢, B, ~ N(0,t), by 
symmetry, E[B,]=0 and E£[B>]=0. Applying the equation for covariance 
Cov(X,Y) = E[XY]- E[X]E[Y], we have Cov(B,,B’) = E[B; ]- E[B, JE[B?]=0-0=0. 
So the correlation of a Brownian motion and its square is 0, too. 


C. Let B, be a Brownian motion. What is the probability that B, > 0 and B, <0? 


Solution: A standard solution takes advantage of the fact that B, ~ N(0,1), and B, — B, 
is independent of B, which is again a normal distribution: B, — B, ~ N(0,1). If 
B, 2 x » 0, then for B, <0, we must have B, — B, « —. 


P(B, > 0, B, < 0) = P(B, > 0, B, — B, < -B)) 
l -x° /2 x l ayta x | ~(x*+y")/2 
= | ——e* “‘dx| ——e” “dy= —e*""*dxd 
a - 179720 ern] i 
s 27 20 0 8 
But do we really need the integration step? If we fully take advantage of the facts that B, 


and B,- B, are two IID N(0,1), the answer is no. Using conditional probability and 
independence, we can reformulate the equation as 


P(B, > 0, B, «0)- P(B, > 0)P(B, — B, «0)P(|B, - B, >| B, |) 
=1/2x1/2x1/2=1/8 


^ W(s)~ N(0,s). So E [exp (AW (s)}] is the moment generating function of normal random variable 
N(O, s). 
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This approach is better demonstrated in Figure 5.10. When we have 8, >0 and 
B, — B, <—B,, which accounts for 1/8 of the density volume. (All 8 regions separated by 
x=0, y 0, y=x, and y - —x have the same density volume by symmetry.) 


0.15 


Figure 5.10 Probability density graph of (B4, Bz-B:) 


Stopping time/ first passage time 


A. What is the mean of the stopping time for a Brownian motion to reach either -1 or 1? 


Solution: As we have discussed, B° —t is martingale. It can be proved by applying Ito’s 
lemma: 


OB) -D ap , BP -O ya | 
0B, ar 0B, 


2 
d(B? -1)- 190 0 = 2B dB, — dt + dt = 2BdB,. 


So d(B; —1) has no drift term and is a martingale. Let T — min(t; B, =1 or -I}. At 
continuous time and space, the following property still applies: A martingale stopped at 
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a stopping time is a martingale! So B} —T is a martingale and E E: -T i = B; -0=0. 
The probability that B, hits 1 or -1 is 1, so B; -1— E[T]= E| B; | EL 


B. Let W(t) be a standard Wiener process and 7, (x » 0) be the first passage time to 
level x (7, = minit; W(t) =x}). What is the probability density function of r, and the 


expected value of 7, ? 


Solution: This is a textbook problem that is elegantly solved using the reflection 
principle, so we will simply summarize the explanation. For any Wiener process paths 
that reach x before ¢ (7, <t), they have equal probability ending above x or below x at 
time £, P(t, xt,W(t) 2 x) P(t, &t,W(t) E x). The explanation lies in the reflection 
principle. As shown in Figure 5.11, for each path that reaches x before ¢ and is at a level 
y above x at time ¢, we can switch the sign of any move starting from r, and the 
reflected path will end at 2x — y that is below x at time ¢. For a standard Wiener process 
(Brownian motion), both paths have equal probability. 


P(t. <t)=P(c, <t,W (t) 2 x)  P(c, <t,W (t) € x)= 2P(v, < t,W (t) 2 x) 


- 2P(W(t) 2 x) «2 [ ae ?t dw 


Ww "m 2) dw 
Let v=—=, we have e" "' =e" ”” and m 
í 


vt 


e" "gy 22-2N(xl Nt)? 


s P(t, soe2 [moe dw=2[) 


Take the derivative with respect to t£, we have 


dP(r, «ty dP(r, <t} d(x/Vt) X al xe" 
pee -2N (xIJt)x 20? > , Vx » 0. 
J^ NF REED OO aaa (x/ E) NT 


From part A, it's easy to show that the expected stopping time to reach either a (æa » 0) 
or -B (f>0) is again E[N|- af. The expected first passage time to level x is 


* If we define (1) = max W(s), then P(r <t) if and only if M(1) > x. Taking the derivative of P(t. <t) 
O<s<t 


with respect to x, we can derive the probability density function of M (1). 
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essentially the expected stopping time to reach either x or —o and E|r,] - xxo» = o. 


Although we have P(t, <00)=2-2N (x / Vo) —], the expected value of r, is oo! 


W(t) 
E A 
H 
i [| 
al 
X GD aD aD mo un uD uD AD AD GD AD AD AD AD ED ED uD uD uD aD aD aD aD an am 


ia can conn ns “na A aaa aa aa a aa a ths ————— 


-——————— 5 


Ko 
«x 
e 
i 
I 
i 
i 
i 
Š 
l 
I 
i 
l 
l 
I 
I 
I 
i 
I 
i 
i 


© 
xN 
~ 


Figure 5.11 Sample path of a standard Weiner process and its reflected path 


C. Suppose that X is a Brownian motion with no drift, i.e. dX(t) = dW (t). If X starts at 0, 
what is the probability that X hits 3 before hitting -5? What if X has drift m, i.e. 
dX (t) = mdt + dW (t)? 

Solution: A Brownian motion is a martingale. Let p, be the probability that the 


Brownian motion hits 3 before -5. Since a martingale stopped at a stopping time is a 
martingale, we have 3P, c (-5)1- 5) 20 >P 25/8. Similar to random walk, if we 


have stopping boundaries (æ » 0) and —f (2 » 0), the probability that it stops at a 
instead of —f is p, = J /(a + D). The expected stopping time to reach either a or -8 
is again E[N |= of. 

When X has drift m, the process is no longer a martingale. Let P(t, x) be the probability 
that the process hits 3 before hitting -5 when X = x at time ¢. Although X is no longer a 
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martingale process, it is still a Markov process. So P(t, x) = P(x) is actually independent 
of t. Applying the Feynman-Kac equation’, we have 


mP (x) -1/2P, (x) 0 for -5«x«3. 
We also have boundary conditions that P(3) 21 and P(—5)=0. 


mP (x) *1/2P, (x) =0 is a homogeneous linear differential equation with two real roots: 


mx 


r =0 and r,=-2m. So the general solution is P(x) 2 ce" +c,e°™ =c *ce^ 
Applying the boundary conditions, we have 
k +c,e°" =1 E ^ = —gl?n j(g 9" —. g19m) e ..] 


=> P(0)2c +c, 2-4 


10m .— E. —6m 10m 0m _ ,-6m 
Cc-c6e -0 c,-l/(e" -e ") e e 


A different and simpler approach takes advantage of the exponential martingale: 
Z(t)= exp [AW (r) -14?1). Since W(t) = X(t) - mt, X(t) — mt is a Brownian motion as 
well. Applying the exponential martingale, we have E | exp(4(X -mt)-iA'r)| =1 for 
any constant A. To remove the terms including time ¢, we can set A — —2m and the 
equation becomes E | exp(-2mX )] =1, Since a martingale stopped at a stopping time is 


10m — 
a martingale, we have P, exp(—2mx3)+ (1- B)exp(-2mx —-5) 21— T 
e" — 


oon : 


D. Suppose that X is a generalized Weiner process dX = dt+dW(t), where W(t) is a 
Brownian motion. What is the probability that X ever reaches -1? 


Solution: To solve this problem, we again can use the equation E| exp(-2mX )| =] 


from the previous problem with m=1. It may not be obvious since we only have one 
apparent boundary, —1. To apply the stopping time, we also need a corresponding 
positive boundary. To address this problem, we can simply use +00 as the positive 
boundary and the equation becomes 


* Let X be an Ito process given by equation dX (t) = (t, X)dt+y(t,X)dW and f(x) be a function of X. 
Define function V (t, x) = E[f(X,)| X, =x], then V(t,x) is a martingale process that satisfies the partial 


| ...0V OV l, av -: 
differential equation x Pire st gi ues = 0 and terminal condition V (T, x) = f(x) for all 
t S 2 : 


X. 
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P exp(-2 x—1)+ (1- P,)exp(-2x4«) = Pe” =1=> P, =e”. 


Ito's lemma 


Ito’s lemma is the stochastic counterpart of the chain rule in ordinary calculus. Let X(t) 
be an Ito process satisfying dX(t)= D(t, X)dt * y(t, X)dW(t), and f(X(t),t) be a 
twice-differentiable function of X (t) and t. Then f/(X(t),t) is an Ito process satisfying 


df lui MM 


= 2 zi 


Drift rate — 


AND cs 


A. Let B, be a Brownian motion and Z, — JLB,. What is the mean and variance of Z, ? Is 


Z, a martingale process? 


Solution: As a Brownian motion, B, ~ N(O, t), which is symmetric about 0. Since Jt iS 
a constant at /, Z,= VB, is symmetric about 0 and has mean O and variance 
t x var(B,) 2 t^. More exactly, Z, ~ N(0, £^). 


Although Z, has unconditional expected value 0, it is not a martingale. Applying Ito’s 
lemma to Z, 2 J£B,, we have dZ, - as, + Z. dt+4x T dt - 1t? B dt e NtdB, . 


For all the cases that B, #0, which has probability 1, the drift term 14 "^B dt is not 


zero.’ Hence, the process Z, - JB, is not a martingale process. 


B. Let W(t) be a Brownian motion. Is W(t)’ a martingale process? 


? A generalized Wiener process dx = a(x,t)dt + b(x,t)dW (t) is a martingale process if and only if the 
drift term has coefficient a(x,t) = 0. 
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Solution: Applying Ito’s lemma to f(W(t), t) -W(ty, we have ao. 


2 
% =o, ES y — 6W (t), and df (W(t), t) 2 3W (t)dt - 3W (t) aW (t). So again for the 
cases W(t) #0, which has probability 1, the drift term is not zero. Hence, W(t) is not a 
martingale process. 


136 


Chapter 6 Finance 


It used to be common for candidates with no finance knowledge to get hired into 
quantitative finance positions. Although this still happens for candidates with specialized 
knowledge that is in high demand, it’s more likely that you are required, or at least 
expected, to have a basic grasp of topics in finance. So you should expect to answer 
some finance questions and be judged on your answers. 


Besides classic textbooks,' there are a few interview books in the market to help you 
prepare for finance interviews.’ If you want to get prepared for general finance problems, 
you may want to read a finance interview book to get a feel for what types of questions 
are asked. The focus of this chapter is more on the intuitions and mathematics behind 
derivative pricing instead of basic finance knowledge. Derivative problems are popular 
choices in quantitative interviews—even for divisions that are not directly related to 
derivative markets—because these problems are complex enough to test your 
understanding of quantitative finance. 


6.1. Option Pricing 
Let’s begin with some notations that we will use in the following sections. 


T: maturity date; t: the current time; r = T —t : time to maturity; S: stock price at time t; 
r: continuous risk-free interest rate; y: continuous dividend yield; o: annualized asset 
volatility; c: price of a European call; p: price of a European put; C: price of an 
American call; P: price of an American put; D: present value, at ¢, of future dividends; K: 
strike price; PV: present value at t. 


Price direction of options 


How do vanilla European/American option prices change when S, K, T, o, r, or D 
changes? 


Solution: The payoff of a call is max(S — K,0) and the payoff of a put is max(K —S,0). 


A European option can only be exercised at the expiration time, while an American 
option can be exercised at any time before maturity. Intuitively we can figure out that 
the price of a European/American call should decrease when the strike price increases 


! For basic finance theory and financial market knowledge, I recommend Investments by Zvi Bodie, Alex 
Kane and Alan J. Marcus. For derivatives, Options, Futures and Other Derivatives by John C. Hull is a 
classic. If you want to gain a deeper understanding of stochastic calculus and derivative pricing, I'd 
recommend Stochastic Calculus for Finance (Volumes | and II) by Steven E. Shreve. 

? For example, Vault Guide to Finance Interviews and Vault Guide to Advanced and Quantitative 
Finance Interviews. 
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since a call with a higher strike has no higher—and sometimes lower—payoff than a call 
with a lower strike. Using similar analyses, we summarize the effect of changing market 
conditions on an option’s value in Table 6.1. 


The impact of time to maturity on the price of a European call/put is uncertain. If there is 
a large dividend payoff between two different maturity dates, a European call with 
shorter maturity that expires before the ex-dividend date may be worth more than a call 
with longer maturity. For deep in-the-money European puts, the one with shorter 
maturity is worth more since it can be exercised earlier (time value of the money). 


Table 6.1 Impact of S, K, 7 , o, r, and D on option prices 


1: increase; |: decrease; ?: increase or decrease 


It is also worth noting that Table 6.1 assumes that only one factor changes value while 
all others stay the same, which in practice may not be realistic since some of the factors 
are related. For example, a large decrease in interest rate often triggers a stock market 
rally and increases the stock price, which has an opposite effect on option value. 


Put-call parity 


Put-call parity: c+ K " = p+S-—D, where the European call option and the European 
put option have the same underlying security, the same maturity 7 and the same strike 
price K. Since p 2 0, we can also derive boundaries for c, $- D—Ke "* <c< S, from 
the put-call parity. 

For American options, the equality no longer holds and it becomes two inequalities: 
S-D-K<C-P<S-K™. 


Can you write down the put-call parity for European options on non-dividend paying 
stocks and prove it? 
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Solution: The put-call parity for European options on non-dividend paying stocks is 
c+K~™ = p+S. We can treat the left side of the equation as portfolio 4—a call and a 
zero-coupon bond with face value K—and the right side as portfolio B—a put and the 
underlying stock, which is a protective put. Portfolio A4 has payoff 
max(S,—K,0)+K =max(S,,K) at maturity T, portfolio B has payoff 
max(K —S,,0)+S, =max(S,,K) at T. Since both portfolios have the same payoff at T 
and no payoff between ¢ and T, the no-arbitrage argument’ dictates that they must have 
the same value at t. Hence, c+ K`” = p+S. 


If we rearrange the put-call parity equation into c- p = S- K "., it will give us different 
insight. The portfolio on the left side of the equation—long a call and short a put—has 
the payoff max(S, — K,0)-max(K —S,,0)=S,—K, which is the payoff of a forward 
with delivery price K. A forward with delivery price K has present value $—K "'. So 
we again have the put-call parity c- p = $-K '.. This expression shows that when the 


strike price K = S” (forward price), a call has the same value as put; when K < S”, a 
call has higher value; and when K > S”, a put has higher value. 


American v.s. European options 


A. Since American options can be exercised at any time before maturity, they are often 
more valuable than European options with the same characteristics. But when the stock 
pays no dividend, the theoretical price for an American call and European call should be 
the same since it is never optimal to exercise the American call. Why should you never 
exercise an American call on a non-dividend paying stock before maturity? 


Solution: There are a number of solutions to this popular problem. We present three 
arguments for the conclusion. 


Argument 1. If you exercises the call option, you will only get the intrinsic value of the 
call S — K. The price of the American/European call also includes time value, which is 
positive for a call on a non-dividend paying stock. So the investor is better off selling the 
option than exercising it before maturity. 


In fact, if we rearrange the put-call parity for European options, we have 
c-S-K" -p-z(S-K)*(K-K ")- p. The value of a European call on a non- 


dividend paying stock includes three components: the first component is the intrinsic 
value S — K; the second component is the time value of the strike (if you exercise now, 


? A set of transactions is an arbitrage opportunity if the initial investment < 0; payoff > 0; and at least one 
of the inequalities is strict. 
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you pay K now instead of K at the maturity date, which is lower in present value); and 
the third component is the value of the put, which is often considered to be a protection 
against falling stock price. Clearly the second and the third components are both positive. 
So the European call should be worth more than its intrinsic value. Considering that the 
corresponding American call is worth at least as much as the European call, it is worth 
more than its intrinsic value as well. As a result, it is not optimal to exercise the 
American call before maturity. 


Argument 2. Let's compare two different strategies. In strategy 1, we exercise the call 
option’ at time ¢ (t < T) and receive cash S — K. Alternatively, we can keep the call, 
short the underlying stock and lend K dollars with interest rate r (the cash proceedings 
from the short sale, S, is larger than K). At the maturity date 7, we exercise the call if it's 
in the money, close the short position and close the lending. Table 6.2 shows the cash 
flow of such a strategy: 


It clearly shows that at time t, we have the same cash flow as exercising the call, S — K. 
But at time 7, we always have positive cash flow as well. So this strategy is clearly 
better than exercising the call at time ¢. By keeping the call alive, the extra benefit can be 
realized at maturity. 


Table 6.2 Payoff of an alternative strategy without exercising the call 


Argument 3. Let's use a mathematical argument relying on risk-neutral pricing and 
Jensen’s inequality—if f(X) is a convex function,” then £[ f(X)]2 fCELX]). From 
Figure 6.1, it's obvious that the payoff (if exercised when SS » K ) of a call option 
C(S) -(S—K)' isa convex function of stock price with property 


C(AS, * (1- A)S,) < AC(S,) + (2 A)C(S,), O< AKI. 


^ We assume S > K in our discussion. Otherwise, the call surely should not be exercised. 
> A function f(X) is convex if and only if f (Ax +(1- Ay) «Af(x)*(1- A)f(y), O«A«lI. If 
f (x) > 0, Vx, then f(X) is convex. 
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Let S, =S and S, =0, then C(AS) < AC(S)-- (1— A)C(0) = AC(S) since C(0) =0. 


AC(Si)*(10-A)C(S;) 
C(AS (0 -A)S2) 


S^ AS;+(1-A)S2 Si S 


Figure 6.1 Payoff of a European call option 


If the option is exercised at time ¢, the payoff at t is C(S, — K). If it is not exercised until 
maturity, the discounted expected payoff (to A is E Le" C(S;) | under risk-neutral 


measure. Under risk-neutral probabilities, we also have E[S,]=S,e”. 
So É|e"C(S,) |-e"E[C(S;)] 2e "C(E[S,]) - e "C(e"S,). 
where the inequality is from Jensen's inequality. 


Let S Ze" S, and A-e" , we have C(AS) = C(S,) «e " C(e"s, ) < e" E[C(S, )] l 


Since the discounted payoff e "^ E [C(S,)] is no less than C(S,) for any t € T under the 
risk neutral measure, it is never optimal to exercise the option before expiration. 
I should point out that the payoff of a put is also a convex function of the stock price. 


But it is often optimal to exercise an American put on a non-dividend paying stock. The 
difference is that P(0) =K, so it does not have the property that P(AS) < AP(S). In fact, 


P(AS) 2 AP(S). So the argument for American calls does not apply to American puts. 
Similar analysis can also show that early exercise of an American call option for 


dividend-paying stocks is never optimal except possibly for the time right before an ex- 
dividend date. 


B. A European put option on a non-dividend paying stock with strike price $80 is 
currently priced at $8 and a put option on the same stock with strike price $90 is priced 
at $9. Is there an arbitrage opportunity existing in these two options? 
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Solution: In the last problem, we mentioned that the payoff of a put is a convex function 
in stock price. The price of a put option as a function of the strike price is a convex 
function as well. Since a put option with strike 0 is worthless, we always have 
P(0)+AP(K)=AP(K)> P(AK). 

For this specific problem, we should have 8/9x P(90)=8/9x9 =8 > P(80). Since the 


put option with strike price $80 is currently price at 8, it is overpriced and we should 
short it. The overall arbitrage portfolio is to short 9 units of put with K = $80 and long 8 
units of put with K = 90. At time 0, the initial cash flow is 0. At the maturity date, we 
have three possible scenarios: 


S, 290, payoff = 0 (No put is exercised.) 
90> S, 280, payoff =8x(90—S,) > 0 (Puts with K = 90 are exercised.) 
S, «80, payoff = 8 x (90 — S,)—9 x (80 — S, ) = S, > 0 (All puts are exercised.) 


The final payoff 2 0 with positive probability that payoff > 0. So it is clearly an 
arbitrage opportunity. 


Black-Scholes-Merton differential equation 


Can you write down the Black-Scholes-Merton differential equation and briefly explain 
how to derive it? 


Solution: If the evolution of the stock price is a geometric Brownian motion, 

dS = uSdt+oSdW (t), and the derivative V 2V(S,t) is a function of S and f, then 

applying Ito’s lemma yields: 

CL eS ; 

OS 2 OS 

The Black-Scholes-Merton differential equation is a partial differential equation that 
OV WV 1 ;0,0V _ 


should be satisfied by V: —+rS—+ —o°S =r. 
Ot OS 2 os 


To derive the Black-Scholes-Merton differential equation, we build a portfolio with two 


2 
i at + os% aw (t), where W(t) is a Brownian motion. 


OV 
dV =(——+ uS 
Ux 4 


components: long one unit of the derivative and short x unit of the underlying stock. 


Then the portfolio has value I1 =V AS and the change of II follows equation 
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dri = dy -Las 
aS 
oV oV 1 QV oV OV 
= (— qnd dt + oS — dW (t) -—(uSdt + oSdW (t 
or as ta? > ag?) as 4^ (0 73s 4 D) 
2 
ô 2 as 


It is apparent that this portfolio is risk-free since it has no diffusion term. It should have 


risk-free rate of return as well: dII =r(V - S Sat Combining these results we have 


2 2 
e — di aH dm — Lm pet iss e á 
os os Ot OS 2 os 


which is the Black-Scholes-Merton differential equation. 


(les Sry, 
Ot 2 


The Black-Scholes-Merton differential equation is a special case of the discounted 
Feynman-Kac theorem. The discounted Feynman-Kac theorem builds the bridge 
between stochastic differential equations and partial differential equations and applies to 
all Ito processes in general: 

Let X be an Ito process given by equation dX (t) = p(t, X)dt + y(t, X)dW (t) and f(x) 
be a function of X. Define function V (t,x) = E[e"" f(X,)| X, =x], then V(t, x) is a 
martingale process that satisfies the partial differential equation 

oV 
Ox? 
and boundary condition V(T,x) = f(x) for all x. 


Under risk-neutral measure, dS =rSdt+oSdW(t). Let S=X, p(t, X) 2rS and 
y(t, X) 08, then the discounted Feynman-Kac equation becomes the Black-Scholes- 


2 
Merton differential equation OK pps AN E. S^ E 4 - rV. 
Ot OS 2 Os 


OV OV 1l., 
— + p(t,x) — -— Y (t, - rV (t, 
a p em 57 ( x) rV (t,x) 


Black-Scholes formula 


The Black-Scholes formula for European calls and puts with continuous dividend yield y 
is: 


c = Se” N(d)-Ke " N(d,) and p= Ke" N(-d,)- Se” N(-d,), 
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_ In(Se~*" /K)+(r+o0° /2)r B In(S/ K)-(r-y *0^/2)r 
ovr ovr 


I Wm. 
d, Tertio o U R 


d, 
where 


N(x) is the cdf of the standard normal distribution and N (x) is the pdf of the standard 


normal distribution: N(x) = Lx e" 2 dy and N'(x)- l e". 
T 


V27 


If the underlying asset is a futures contract, then yield y =r. If the underlying asset is a 
foreign currency, then yield y =r, , where r, is the foreign risk-free interest rate. 


A. What are the assumptions behind the Black-Scholes formula? 


Solution: The original Black-Scholes formula for European calls and puts consists of the 
equations c= SN(d,) - Ke" N(d,) and p= Ke " N(-d,) - SN(-d,), which require the 
following assumptions: 

1. The stock pays no dividends. 

2. The risk-free interest rate is constant and known. 


3. The stock price follows a geometric Brownian motion with constant drift u and 
volatility o: dS = uSdt + oSdW (t). 


4. There are no transaction costs or taxes; the proceeds of short selling can be fully 
invested. 


5. All securities are perfectly divisible. 


6. Thereare no risk-free arbitrage opportunities. 


B. How can you derive the Black-Scholes formula for a European call on a non-dividend 
paying stock using risk-neutral probability measure? 


Solution: The Black-Scholes formula for a European call on a non-dividend paying 
stock is 
In(S / K) -(r- o^ /2)c 


and d, = d ~ovr. 
ovr 2 l 


c-SN(d)-Ke ™ N(d,), where d, = 
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Under the risk-neutral probability measure, the drift of stock price becomes the risk-free 
interest rate r(t): dS =r(t)Sdt+oSdW (t). Risk-neutral measure allows the option to be 


priced as the discounted value of its expected payoff with the risk-free interest rate: 


/O-E|e 


T 
d "vaso | 0<t<T, where V(T) is the payoff at maturity T. 


When r is constant, the formula can be further simplified as V(t) Ze "E It SO |. 
Under risk-neutral probabilities, dS = rSdt + oSSdW (t). Applying Ito's lemma, we get 


d (In(S)) = ( -o? /2)dt + odW (t) => In S, ~ N (In S € (r—6? /2)z, oc). 
So S, = Selo! Dr94:5 where £~ N (0, 1). For a European option, we have 


VC) B Se 70 120a te ES if Sell ^ /2)ro dre >K 
: otherwise 
In(K / S) - (r6? /2)c 


ox 


E|V(T) | S] = E|max(S, — K,0)| S] = [, eens -K|-——e "de 
2 


Sel" /2)r-o dre 


»Kog» = —d, and 


a 
e ? de 


l e-vro} 12 l 
= Se" | —~e de — K 
h J27 f, V 20 
Let €=e-oVr, then de=dé, e=-d, > &=-d, —ovVt =—d, and we have 


2 
cre = Se" [ v e* de -Se"N(d,), 
l T 


rT l -(é 
Se Lh T e 


K | ee? de = K (1-N(-d,)) = KN (d,) 


J27 
-. E(V(T)| = Se" N(d,) -KN(d,) and V()-e"E|V(T)]- SN (d,)-Ke™N(d,) 


From the derivation process, it is also obvious that 1- N(-d,) = N(d,) is the risk- 
neutral probability that the call option finishes in the money. 


C. How do you derive the Black-Scholes formula for a European call option on a non- 
dividend paying stock by solving the Black-Scholes-Merton differential equation? 
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Solution: You can skip this problem if you don’t have background in partial differential 
equations (PDE). One approach to solving the problem is to convert the Black-Scholes- 
Merton differential equation to a heat equation and then apply the boundary conditions 
to the heat equation to derive the Black-Scholes formula. 


Let y=InS (S =e’) and =T -t, then 2 NE: oV _ OV dy © 19V 


a OF OS dydS Say 
(He) Lr) -18V 18V , 


= — —— | = —— + — —— + — 
aS? aS. as OS | S Oy S?^0y S OS\ Oy S^ dy ts oy 
2 
The Black-Scholes-Merton differential equation C a lgs E a, =0 
Ot oS 2 OS 
can be converted to - Harder E. z E a -rV =0. 
OT 2 oy 27 oy! 
2 
Let u=e"V , the equation becomes rinm stoto, 
Ot 2 Oy 2 Oy 
Finally, let x=y+(r-10 rns (r- 2e ji and 7-7, then es and 
2 2 Oy Ox 
Harta" B. which transforms the equation to 
OT OT 2 Ox 
Ou | ] | | ] m | ,0'u Ou |] ,6€u 
S| =o". Lp Fo ar —, -0 >= _o 
OT 2 Ox 2 ox 2 Ox OT 2 cx 
- l T . u l ,6'u 
So the original equation becomes a heat/diffusion equation 2.759 zo For heat 
T x 
. ðu l ,6€u 
equation prm 22 22 where u 2 u(x,7) is a function of time r and space variable x, 
T x 


with boundary condition 4(x,0) 2 4,(x), the solution is 


2 
l _ 
u(x, T) = Dro fu (w)exp pex. 1 


° The log is taken to convert the geometric Brownian motion to an arithmetic Brownian motion; 7 = 7 — t 
Is used to convert the equation from a backward equation to a forward equation with initial condition at 
T = 0 (the boundary condition at f= T — 7 =0). 
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For European calls, the boundary condition is u,(S,) = max (S, —K, 0). 


S = exp(x-(r-0.507)r). When x=y and T =0, S, =e”. 


2 
u(S,7) =u(x,T) = ILL f max (e" -rojol ET. hy 


2 
l v (x-y) 
EL ul. aK oe iai 
V22t0 Ls " e| 2c |^ 


2 
Let e=, then de= ay ieee, -EÙ =e%'? and when 


ovr’ ovr’ 2o0°r 
_ In(K/S)-(r-07 /2)r _ 


oc 
| 


- u(S,t) = [. (set imet K) 
2 7 


y —InK, € -d, 


_ n2 
e E 2de 


Now, it’s clear that the equation for u(S,r) is exactly the same as the equation for 
E|V(T)|S] in question B. Hence, we have V(S,t) -e "u(S,7) - SN(d) - Ke " N(d,) 


as well. 


D. Assume zero interest rate and a stock with current price at $1 that pays no dividend. 
When the price hits level $H ( H >1) for the first time you can exercise the option and 
receive $1. What is this option worth to you today? 


Solution: First let's use a brute-force approach to solve the problem by assuming that 
the stock price follows a geometric Brownian motion under risk-neutral measure: 
dS -rSdt - oSdW (t). Since r=0, dS=oSdW(t)=> d(In S) 2 -1o*dt - odW(t). 
When ¢ =0, we have S, 21» In(S,) =0. 


Ou 10°u 
"The fundamental solution to heat equation ae ae ; with initial condition u,(w) = f(y) is 
T 20x 


l 2 
u(x,t) = NC =x, =W)f(w)dy, where p(x = x|x, =w)= Mrs exp|-(x-y) /21]. 
Zt 


For detailed discussion about heat equation, please refer to The Mathematics of Financial Derivatives by 
Paul Wilmott, Sam Howison, and Jeff Dewynne. 


147 


Finance 


In $ - ot ! | 
— ——À——- =W (t) is a Brownian motion. 


Hence, InS=-1to*t+oW(t)=> 
Whenever S reaches $H, the payoff is $1. Because the interest rate is 0, the discounted 
payoff is also $1 under risk-neutral measure. So the value of the option is the probability 
that S ever reaches $H, which is equivalent to the probability that In ever reaches 


In H. Again we can apply the exponential martingale Z(t) = exp [AW (t) -1 A?t| as we 
(Moca 
did in Chapter 5: E[Z(t)|= El ex [aes e -]. 
o 


To remove the terms including time ft, we can set A=o and the equation becomes 
E | exp(In S) | =1. The Let P be the probability that InS ever reaches In H (using —o 


as the negative boundary for stopping time), we have 
Pexp(In H) *- (1- P)exp(—œ)= Px H =1> P=1/H. 


So the probability that S ever reaches $H is 1/H and the price of the option should be 
$1/H. Notice that S is a martingale under the risk-neutral measure;? but In $ has a 
negative drift. The reason is that In S. follows a (symmetrical) normal distribution, but S 
itself follows a lognormal distribution, which is positively skewed. As T — oo, although 


the expected value of S, is 1, the probability that S, 21 actually approaches 0. 


It is simpler to use a no-arbitrage argument to derive the price. In order to pay $1 when 
the stock price hits $H, we need to buy 1/H shares of the stock (at $1/H). So the option 
should be worth no more than $1/H. Yet if the option price C is less than $1/H 
(C «1/ H 2 CH <1), we can buy an option by borrowing C shares of the stock. The 
initial investment is 0. Once the stock price hits $77, we will excise the option and return 
the stock by buying C shares at price $H, which gives payoff 1 — CH » 0. That means we 
have no initial investment, yet we have possible positive future payoff, which is 
contradictory to the no arbitrage argument. So the price cannot be less than $1/H. Hence, 
the price is exactly $1/H. 


E. Assume a non-dividend paying stock follows a geometric Brownian motion. What is 
the value of a contract that at maturity 7 pays the inverse of the stock price observed at 
the maturity? 


* Once we recognize that S is a martingale under the risk neutral measure, we do not need the assumption 
that S follows a geometric Brownian motion. S has two boundaries for stopping: 0 and H. The boundary 
conditions are /(0) 2 0 and f(//) - 1. Using the martingale, the probability that it will ever reaches H is 


Px H *(1-P)x02 S, 212 P-1/ H. 
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Solution: Under risk-neutral measure d$ =rSdt+oaSdW(t). Apply Ito's lemma to 


2 
dV = o ge o ^s? d+ osdw(t) 
y - l. Os ot 208 os 
S l | 23: x l j 
= E i S dicc OSG a )Vdt - oVdW (t) 


So V follows a geometric Brownian motion as well and we can apply Ito's lemma to 
InV: 


V ; lV’? , V l , 
d(InV )= DUREE aug d r2 Ed dt - odW (t). 


Hence, In(V,) ~ In(V,)+N((-r+407)t,0°r) and E[V,]- E[e"^]- Le 77". 


Discounting the payoff by €", we have V =e" E|V,.|- dg uim 


6.2. The Greeks 


All Greeks are first-order or second-order partial derivatives of the option price with 
respect to different underlying factors, which are used to measure the risks—as well as 
potential returns—of the financial derivative. The following Greeks for a derivative fare 
routinely used by financial institutions: 


2 
Delta: EECA Gamma: [= E f ; Theta: Dg. Vega: pese Rho: asp 
OS OS Ot Qo or 


Delta 
For a European call with dividend yield y: A =e” N(d,) 


For a European put with dividend yield y: A = —e "[1— N(d,)] 


A. What is the delta of a European call option on a non-dividend paying stock? How do 
you derive the delta? 


Solution: The delta of a European call on a non-dividend paying stock has a clean 
expression: A= N(d,). For the derivation, though, many make the mistake by treating 
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N(d,)and N(d,) as constants in the call pricing formula c = SN (d))- Ke" N (d) and 
simply taking the partial derivative on S to yield N(d,). The derivation step is actually 
more complex than that since both N(d,) and N(d,) are functions of S through d, and 
d,. So the correct partial derivative is x = N(d,)+ Sx Ud N(d)-Ke"* ae N(d,). 
os Os OS 


Take the partial derivative with respect to S for N(d,) and N (d,)”: 


ð Ó ] -d?/2 l l -d?/2 
— N(d))- N'(d —d -—e ^x = —— e “I 
os | Jas | J27 SoVt Sov2atr 
o Ó l -d2 /2 l l -(d -aJry /2 
CAR Bde EE LL ee 
os "0S * 2m Sodr | SoX2zr 
- 1]. edt "Aue rd garl 8 l od? pig S E 
Soxv2zc Sov2at K 


ð Sort ð -rr Ó 
So we have EC s DAG Teta Ane ae ay) ©: Hence, the 


last two components of 2 cancel out and ce = N(d,). 
OS oS 


B. What is your estimate of the delta of an at-the-money call on a stock without dividend? 
What will happen to delta as the at-the-money option approaches maturity date? 


Solution: For an at-the-money European call, the stock price equals the strike price. 
(r+0°/2)r 


ovr 


6.2, all at-the-money call options indeed have A>0.5 and the longer the maturity, the 
higher the A. As T-1— 0, (—+ vr +0 = N(d,) = N(0)=0.5, which is also 
o 


S=K>d,= -C Wr > 0 and A= N(d,) » 0.5. As shown in Figure 
o 


shown in Figure 6.2 (T =10 days). The same argument is true for calls on stock with 
continuous dividend rate y if r> y. 


Figure 6.2 also shows that when S is large (S >> K), A approaches |. Furthermore, the 
shorter the maturity, the faster the delta approaches 1. On the other hand, if S is small 
(S << K ), A approaches 0 and the shorter the maturity, the faster the delta approaches 0. 


= ôd, 


id ' S r—y)t t Od 
d, =d, -ovt = N (d,)= e n N (d), e 
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Delta of Call Options 


0.9 t=10days fat 
--—-—-—-— t= 1 month P. 
0.8 RRRBBARRRE = 3 months J : 
0.7 
0.6 " 
————rA——Ó—IITIITIII—Im P, 
X9 
© 0.5 
O 
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Figure 6.2 Variation of delta of a European call option with respect to S and 
T. K= 100, r = 0.05, o = 0.25. 


C. You just entered a long position for a European call option on GM stock and decide 
to dynamically hedge the position to eliminate the risk from the fluctuation of GM stock 
price. How will you hedge the call option? If after your hedge, the price of GM has a 
sudden increase, how will you rebalance your hedging position? 


In(S/K)+(r—y+o?/2)r 
ovr 


increasing function of d,, we have ST=>d,T=> AT. 


Solution: Since d, = and A=e ’* N(d,) is a monotonously 


One hedging method is delta hedging, for which we short A = e "" N(d,) shares of stock 


for each unit of call option to make the portfolio delta-neutral. Since A shares of GM 
stock costs more than one unit of GM option, we also need to invest cash (if the option 


price exactly follows the Black-Scholes formula, we need to lend $Ke '" N(d,) for each 
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unit of option) in the money market. If there is a sudden increase in S, d, increases and 
A increases as well. That means we need to short more stock and lend more cash 
(Ke ™ N(d,) also increases). 


The delta hedge only replicates the value and the slope of the option. To hedge the 
curvature of the option, we will need to hedge gamma as well. 


D. Can you estimate the value of an at-the-money call on a non-dividend paying stock? 
Assume the interest rate is low and the call has short maturity. 


Solution: When S-K, we have c=S ( N(d,)-e "N (d,)). In a low-interest 
environment, r ~ 0 and e" «1,so c= S(N(d)) - N(d,)). 


We also have N(d,)—N(d,)= [mea 
2 2 


where d, aN and d, = (o E 
go 2 o 2 


For a small r, a typical o for stocks (« 40% per year) and a short maturity (< 3 months), 
both d, and d, are close to 0. For example, if r = 0.03, o = 0.3, and 7 21/6 year, then 


d, = —0.02 and e? Ë =0.98. 


l ovr 
-. N(d )- N(d,) = —=(d, -d,) == 040VT -t >c 2 0.40 NT . 
l 2 Pr l 2 Nr 
In practice, this approximation is used by some volatility traders to estimate the implied 
volatility of an at-the-money option. 


—1/2x* -p2x? 


<1; but the 


approximation —e ™ K x -K causes a small underestimation. To some extent, the two 
opposite effects cancel out and the overall approximation is fairly accurate.) 


(The approximation e ~ 1 causes a small overestimation since e 


Gamma 


Sat eee N (d, )e ” 
For a European call/put with dividend yield y: Ir = ————— 
So 
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What happens to the gamma of an at-the-money European option when it approaches its 
maturity? 


Solution: From the put-call parity, it is obvious that a call and a put with identical 
characteristics have the same gamma (since I' =0 for both the cash position and the 


underlying stock). Taking the partial derivative of the A of a call option with respect to 
! —yt 2 

SAC d , where N (d,) = gd 

Sovr 


J2z 


So for plain vanilla call and put options, gamma is always positive. 


S, we have [ = 


Figure 6.3 shows that gamma is high when options are at the money, which is the stock 
price region that A changes rapidly with S. If S «« K or S >> K (deep in the money or 
out of the money), gamma approaches 0 since A stays constant at | or 0. 


The gamma of options with shorter maturities approaches 0 much faster than options 
with longer maturities as $ moves away from K. So for deep in-the-money or deep out- 
of-the-money options, longer maturity means higher gamma. In contrast, if the stock 
prices are close to the strike price (at the money) as the maturity nears, the slope of delta 
for an at-the-money call becomes steeper and steeper. So for options close to the strike 
price, shorter-term options have higher gammas. 


As T — 0, an at-the-money call/put has I' — œ (A becomes a step function). This can 
be shown from the formula of gamma for a European call/put with no dividend, 


N'4). 
I" 


When S = K, d, = lim(— + svt >0> lim N (d) > LE The numerator is 1/42z; 
TWO T 


V27 


yet the denominator has a limit lim Sec — 0, so I > oo. In other words, When t=T, 
T 


delta becomes a step function. This phenomenon makes hedging at-the-money options 
difficult when ¢ — T since delta is extremely sensitive to changes in S. 
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Gamma of Call/Put Options 
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Figure 6.3 Variation of gamma of a European call option with respect to S and T. 
K = 100, r= 0.05, 0 = 0.25. 


Theta 
' —yr 
For a European call option: © = Be ie CP ici aia ySe " N(d)-rKe " N(d,) 
2r 
' —yr 
For a European put option: © = ECUADOR ySe " N(-d)4rKe " N(-d,) 
24r 
When there is no dividend, the theta for a European call option is simplified to 
© = A rkeN (d,), which is always negative. As shown in Figure 6.4, when 
T 


S «« K, N(d,)~0 and N'(dj)«0. Hence, @>0. When S>>K, N(d,)«1 and 
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N'(d,)*0. Hence, O 2 -rKe ". When S« K, © has large negative value and the 


smaller the r, the more negative the ©. 


Theta 
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Figure 6.4 Variation of theta of a European call option with respect to S and 
T. K = 100,072 0.25, r=0.05 


A. When will a European option have positive theta? 


Solution: For American options as well as European calls on non-dividend paying assets, 
theta is always negative. But for deep in-the-money European puts, their values may 
increase as ¢ approaches T if all other factors remain the same, so they may have positive 


theta. 


SN (d) 


A put option on a non-dividend paying asset has © = ux t rKe " N(-d,). If the 
T 


put option is deep in-the-money (S «« K), then N'(d,)~0 and N(-d,)~1. Hence, 
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© s rKe "* » 0. That's also the reason why it can be optimal to exercise a deep in-the- 
money American put before maturity. 


For deep in-the-money European call options with high dividend yield, the theta can be 
positive as well. If a call option with high dividend yield is deep in-the-money (S >> K ), 


N(d)sN(d,)zsl N'(d,)«0, so the component ySe "* N(d,) can make © positive. 


B. You just entered a long position for a call option on GM and hedged the position by 
shorting GM shares to make the portfolio delta neutral. If there is an immediate increase 
or decrease in GM's stock price, what will happen to the value of your portfolio? Is it an 
arbitrage opportunity? Assume that GM does not pay dividends. 


Solution: A position in the underlying asset has zero gamma. So the portfolio is delta- 
neutral and long gamma. Therefore, either an immediate increase or decrease in the GM 
stock price will increase the portfolio value. The convexity (positive gamma) enhances 
returns when there is a large move in the stock price in either direction. 


Nevertheless, it is not an arbitrage opportunity. It is a trade-off between gamma and 
theta instead. From the Black-Scholes-Merton differential equation, the portfolio V 


OV | V 


2 
satisfies the equation £, r$ —+-—o°S’ id =@+rSA+ 5 o°S'T =rV. For a delta- 


oS 2 oS 
neutral portfolio, we have © + 2055 —T-rV. This indicates that gamma and theta often 


have opposite signs. For example, when an at-the-money call approaches maturity, 
gamma is large and positive, so theta is large and negative. Our delta neutral portfolio 
has positive gamma and negative theta. That means if the price does not move, the 
passage of time will result in a lower portfolio value unless we rebalance. So the 
portfolio does not provide an arbitrage opportunity. 


Vega 


For European options: v = Ls = id = Se" JN (d,) 

Oo Oo 
At-the-money options are most sensitive to volatility change, so they have higher vegas 
than either in-the-money or out-of-the-money options. The vegas of all options decrease 
as time to expiration becomes shorter ( Jr 0 ) since a long-term option is more 
sensitive to change in volatility. 


A. Explain implied volatility and volatility smile. What is the implication of volatility 
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smile for the Black-Scholes pricing model? 


Solution: Implied volatility is the volatility that makes the model option price equal to 
the market option price. Volatility smile describes the relationship between the implied 
volatility of the options and the strike prices for a given asset. For currency options, 
implied volatilities tend to be higher for in-the-money and out-of-the-money options 
than for at-the-money options. For equity, volatility often decreases as the strike price 
increases (also called volatility skew). The Black-Scholes model assumes that the asset 
price follows a lognormal distribution with constant volatility. In reality, volatilities are 
neither constant nor deterministic. In fact, the volatility is a stochastic process itself. 
Furthermore, there may be jumps in asset prices. 


B. You have to price a European call option either with a constant volatility 30% or by 
drawing volatility from a random distribution with a mean of 30%. Which option would 
be more expensive? 


Solution: Many would simply argue that stochastic volatility makes the stock price 
more volatile, so the call price is more valuable when the volatility is drawn from a 
random distribution. Mathematically, the underlying argument is that the price of a 
European call option is a convex function of volatility and as a result 


c(E[o}) <E [c(o)], where o is the random variable representing volatility and c is the 


call option price. Is the underlying argument correct? It’s correct in most, but not all, 


2 
cases. If the call price c is always a convex function of o, then - - > 0. x is the 
Oo o 


Vega of the option. For a European call option, 


óc S Jc 
p =——=SVrN (d) 2 — exp(-d! /2). 
aa TN ) Jon xp( l ) 


2 
The secondary partial derivative — is called Volga. For a European call option, 
2 
E - - Syr exp(-d; i2) 4e. p 
Oo” 2z o o 


v is always positive. For most out-of-the-money call options, both d, and d, are 
negative; for most in-the-money call options, both d, and d, are positive. So dd, > 0 


in most cases and c is a convex function of o when d,d, > 0. But theoretically, we can 


Ó'c 


o? 


have conditions that d, » 0 and d, <0 and <0 when the option is close to being 
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at-the-money. So the function is not always convex. In those cases, the option with 
constant volatility may have a higher value. 


C. The Black-Scholes formula for non-dividend paying stocks assumes that the stock 
follows a geometric Brownian motion. Now assume that you don’t know the stochastic 
process followed by the stock price, but you have the European call prices for all 
(continuous) strike prices K. Can you determine the risk-neutral probability density 
function of the stock price at time 7? 


Solution: The payoff a European call at its maturity date is Max(S, — K, 0). Therefore 


under risk-neutral measure, we have c-e"* [G-K fs (s)ds, where f, (s) is the 


probability density function of S, under the risk-neutral probability measure. Taking the 
first and second derivatives of c with respect to K, we have 


óc. 
X ser 6-105, G8 
-rT [36-5 p oaea 
=e" [ -fs ()ds 
Ü'c o óc r = — „rT 
OK’ -2(£)- x OK k fs, (s)ds =e ts (K) . 


Ó'c 


Hence the risk-neutral probability density function is f, (K)- e" ak? 


6.3. Option Portfolios and Exotic Options 


In addition to the pricing and properties of vanilla European and American options, you 
may be expected to be familiar with the construction and payoff of basic option-based 
trading strategies—covered call, protective put, bull/bear spread, butterfly spread, 
straddle, etc. Furthermore, if you are applying for a derivatives-related position, you 


1° To calculate the derivatives requires the Leibniz integral rule, a formula for differentiating a definite 
i whose limits are functions of the — variable: 


= = [re z)dx = oe E AE 2- fac) 2 
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should also have a good understanding of pricing and hedging of some of the common 
exotic derivatives—binary option, barrier option, Asian option, chooser option, etc. 


Bull spread 


What are the price boundaries for a bull call spread? 


Solution: A bull call spread is a portfolio with two options: long a call c, with strike K, 
and short a call c, with strike K, (K, < K,). The cash flow of a bull spread is 
summarized in table 6.3. 


Time 0 


Cash flow 


K <5, eK 


Table 6.3 Cash flows of a bull call spread. 


Since K, < K,, the initial cash flow is negative. Considering that the final payoff is 
bounded by K,-K,, the price of the spread, c, — c,, is bounded by e" (K, —K,). 


,-K 


Besides, the payoff is also bounded by A S+, so the price is also bounded by 


2 


K-K g 


2 


Straddle 


Explain what a straddle is and when you want to purchase a straddle. 


Solution: A straddle includes long positions in both a call option and a put option with 
the same strike price K and maturity date T on the same stock. The payoff of a long 
straddle is | S$, — K |. So a straddle may be used to bet on large stock price moves. In 
practice, a straddle is also used as a trading strategy for making bets on volatility. If an 
investor believes that the realized (future) volatility should be much higher than the 
implied volatility of call and put options, he or she will purchase a straddle. For example, 
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the value of an at-the-money call or put is almost a linear function of volatility. If the 
investor purchases an at-the-money straddle, both the call and the put options have the 


price c « px0.40,SVr, where o; is the implied volatility. If the realized volatility 
C, » 0,, both options are undervalued. When the market prices converge to the prices 
with the realized volatility, both the call and the put will become more valuable. 


Although initially a straddle with an at-the-money call and an at-the-money put (K = S ) 
has a delta close to 0, as the stock price moves away from the strike price, the delta is no 
longer close to 0 and the investor is exposed to stock price movements. So a straddle is 
not a pure bet on stock volatility. For a pure bet on volatility, it is better to use volatility 


swaps or variance swaps. For example, a variance swap pays N x(o; — K), where N 


is the notional value, o? 


r 


is the realized variance and K,, is the strike for the variance. 


r 


Binary options 


What is the price of a binary (cash-or-nothing digital) European call option on a non- 
dividend paying stock if the stock price follows a geometric Brownian motion? How 
would you hedge a cash-or-nothing call option and what's the limitation of your hedging 
strategy? 


Solution: A cash-or-nothing call option with strike price K pays $1 if the asset price is 
above the strike price at the maturity date, otherwise it pays nothing. The price of the 


option is c, =e " N(d,) if the underlying asset is a non-dividend paying stock. As we 
have discussed in the derivation of the Black-Scholes formula, N(d,) is the probability 
that a vanilla call option finishes in the money under the risk-neutral measure. So its 
discounted value is e " N(d,). 


Theoretically, a cash-or-nothing call option can be hedged using the standard delta 
Oc l 
hedging strategy. Since A = —--e"N'(d;,)——-, a long position in a cash-or- 
gıng 8y as P em Bp 


nothing call option can be hedged by shorting e” N (= shares (and a risk-free 
ONT 


money market position). Such a hedge works well when the difference between S and K 
is large and 7 is not close to 0. But when the option is approaching maturity T (t > 0) 


'' For detailed discussion about volatility swaps, please refer to the paper “More Than You Ever Wanted 
to Know about Volatility Swaps” by Kresimir Demeterfi, et al. The paper shows that a variance swap can 
be approximated by a portfolio of straddles with proper weights inversely proportional to 1/k’. 


160 


A Practical Guide To Quantitative Finance Interviews 


and the stock price S is close to K, A is extremely volatile" and small changes in the 
stock price cause very large changes in A. In these cases, it is practically impossible to 
hedge a cash-or-nothing call option by delta hedging. 


We can also approximate a digital option using a bull spread with two calls. If call 
options are available for all strike prices and there are no transaction costs, we can long 
1/22 call options with strike price K —& and short 1/2¢ call options with strike price 
K +€. The payoff of the bull spread is the same as the digital call option if S, < K -€ 


(both have payoff 0) or S, 2 K e (both have payoff $1). When K-e<S,<K+e, 
their payoffs are different. Nevertheless, if we set ¢ — 0, such a strategy will exactly 


replicate the digital call. So it provides another way of hedging a digital call option. This 
hedging strategy suffers its own drawback. In practice, not all strike prices are traded in 
the market. Even if all strike prices were traded in the market, the number of options 
needed for hedging, 1/22, will be large in order to keep € small. 


Exchange options 


How would you price an exchange call option that pays max (S;., ze 0) at maturity. 
Assume that S, and S, are non-dividend paying stocks and both follow geometric 


Brownian motions with correlation p. 


Solution: The solution to this problem uses change of numeraire. Numeraire means a 
unit of measurement. When we express the price of an asset, we usually use the local 
currency as the numeraire. But for modeling purposes, it is often easier to use a different 
asset as the numeraire. The only requirement for a numeraire is that it must always be 
positive. 


The payoff of the exchange option depends on both S,, (price of S, at maturity date T) 


and S., (price of S, at T), so it appears that we need two geometric Brownian motions: 
aS, = uS dt + o,S,dW,, 
dS, = 15S, dt + o, S,dW, , 


Yet if we use S, as the numeraire, we can convert the problem to just one geometric 


S 
Brownian motion. The final payoff is max (S; -$,,,0) zr sa i When 


Tl 


-rT 


N um dn 


2 §_+K and t 9 0— In(S/ K) — 0 — d, EE T A E ee eee 
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S, and S, are geometrical Browian motions, f = ra Is a geometric Brownian motion as 
l 


well. One intuitive explanation is that both In S, and In S, follow normal distributions, 


so In f 2In S, - In S, follows a normal distribution as well and f follows a lognormal 


distribution. More rigorously, we can apply the Ito's lemma to f — 25. 
l 


Of 28, Of Of - 


af -S of _ E 


| 
ôS, S?’ ôS, S? S? S?’ aS? °’ SAS, S 


2 2 
af =L as +2 as, QE I E E f dS dS, 
as, ' aS, 2 08, 2 08, AS, aS, 


S, S S S S S 
- —u, —- dt - o, - dW, , * u, dt + o, — dW. , +0, —- dt - poo, — dt 
Hı S l 5, nı 445 S 2 S 1,2 S, POO, S 


l 


= (4 — L, to: - pa,o; ) fat - o, faW,, t o, fdW, , 


~ (u, -4 *9, - po,o, ) fat +o; -2po,0; +0; x fdW,, 
5; 


~| § 
To make f= > a martingale, set 14, — 44 +0, — poo, =0 and we have E ža = 2 
l T, l 


P 
and —- is a martingale under the new measure. The value of the exchange option using 
t,l 


- S 
S, as the numeraire is C, = £ i» wo 2! which is just the value of a call option 


T1 


with underlying asset price S = es strike price K =1, interest rate r = 0, and volatility 
l 


O, = Jo? -2p0,0, +0;. So its value is C, = 3 N(d,)~ N(d;), where 
l 


In (S,/S,)+0.505T 
ovr 


expressed in local currency is S,C, = S,N(d,)—S,N(d,). 


d, = and d, =d, ~ovr. The payoff of the exchange option 
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6.4. Other Finance Questions 


Besides option pricing problems, a variety of other quantitative finance problems are 
tested in quantitative interviews as well. Many of these problems tend to be position- 
specific. For example, if you are applying for a risk management job, prepare to answer 
questions about VaR; for fixed-income jobs, get ready to answer questions about interest 
rate models. As I explained in Chapter 1, it always helps if you grasp the basic 
knowledge before the interview. In this section, we use several examples to show some 
typical interview problems. 


Portfolio optimization 


You are constructing a simple portfolio using two stocks A and B. Both have the same 
expected return of 12%. The standard deviation of A’s return is 20% and the standard 
deviation of B’s return is 30%; the correlation of their returns is 50%. How will you 
allocate your investment between these two stocks to minimize the risk of your portfolio? 


Solution: Portfolio optimization has always been a crucial topic for investment 
management firms. Harry Markowitz’s mean-variance portfolio theory is by far the most 
well-known and well-studied portfolio optimization model. The essence of the mean- 
variance portfolio theory assumes that investors prefer (1) higher expected returns for a 
given level of standard deviation/variance and (2) lower standard deviations/variances 
for a given level of expected return. Portfolios that provide the minimum standard 
deviation for a given expected return are termed efficient portfolios. The expected return 
and the variance of a portfolio with N assets can be expressed as 


T 

Hy, = WH FW test Wy Hy =W U 

N 
var(r,) = 9 0; w, + o ww, =w Dw 

i=] it] 
where w,, Vi 21,---, N , is the weight of the i-th asset in the portfolio; 4, Vi 2 1,---, V, is 
the expected return of the i-th asset; o; is the variance of i-th asset's return; 
O; = 950,0; is the covariance of the returns of the i-th and the j-th assets and p; is their 
correlation; w is an N x1 column vector of w,’s; u is an Nx1 column vector of 4,’s; 
>» is the covariance matrix of the returns of N assets, an NxN matrix. 


Since the optimal portfolio minimizes the variance of the return for a given level of 
expected return, the efficient portfolio can be formulated as the following optimization 
problem: 
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min w’ =w . a 

oe : , where e is an N xl vector with all elements equal to 1. 
st. W u- u,,w e-l 
For this specific problem, the expected returns are 12% for both stocks. So 14, is always 
12% no matter what w, and w, (w, +w, =1) are. The variance of the portfolio is 
var(r,) - 04W, + O5W; * 20, pO ,05W4W; 


2212 2 2 
-O04W, tOs(l-w,) *2p, 509 ,05W,(l— w,) 


Taking the derivative of var(r,) with respect to w, and setting it to zero, we have 


Ó var(r, ) s ; 
are ow, —20,(1—w,)+2p, ,0,0,(1—w,)-2p, 50 ,05W, =90 
A 
0, — P,50405 0.09 —0.5x 0.2x0.3 6 
Eod mre c SS 


^ g!-2p,,0,0,*01 0.04-2x0.5x0.2x034 0.00. 7 


So we should invest 6/7 of the money in stock A and 1/7 in stock B. 


Value at risk 


Briefly explain what VaR is. What is the potential drawback of using VaR to measure 
the risk of derivatives? 


Solution: Value at Risk (VaR) and stress test—or more general scenario analysis—are 
two important aspects of risk management. In the Financial Risk Manager Handbook," 
VaR is defined as the following: VAR is the maximum loss over a target horizon such 
that there is a low, pre-specified probability that the actual loss will be larger. 


Given a confidence level @e(0,1), the VaR can be implicitly defined as 
a= B E (x)dx, where x is the dollar profit (loss) and f(x) 1s its probability density 


function. [n practice, œ is often set to 95% or 99%. VaR is an extremely popular choice 
in financial risk management since it summarizes the risk to a single dollar number. 


C-u B 
3 The optimal weights have closed form solution w*- AX ec yX p, where A = ——F—, 


H,A-B ' -| ' i ' i M 2 
2—————, A-ze'X e>0, B=e'X u, C=u4'} u>0, D=AC-B. 


^ Financial Risk Manager Handbook by Phillippe Jorion is a comprehensive book covering different 
aspects of risk management. A classic book for VaR is Value at Risk, also by Philippe Jorion. 


164 


A Practical Guide To Quantitative Finance Interviews 


Mathematically, it is simply the (negative) first or fifth percentile of the profit 
distribution. 


As a percentile-based measure on the profit distribution, VaR does not depend on the 
shape of the tails before (and after) probability 1 — o, so it does not describe the loss on 


the left tail. When the profit/loss distribution is far from a normal distribution, as in the 
cases of many derivatives, the tail portion has a large impact on the risk, and VaR often 
does not reflect the real risk." For example, let's consider a short position in a credit 
default swap. The underlying asset is bond A with a $1M notional value. Further assume 
that A has a 3% default probability and the loss given default is 100% (no recovery). 
Clearly we are facing the credit risk of bond A. Yet if we use 95% confidence level, 
VaR(A) =0 since the probability of default is less than 5%. 


Furthermore, VaR is not sub-additive and is not a coherent measure of risk, which 
means that when we combine two positions A and B to form a portfolio C, we do not 
always have VaR(C) € VaR(A) * VaR(B). For example, if we add a short position in a 


credit default swap on bond B with a $1M notional value. B also has a 3% default 
probability independent of A and the loss given default is 100%. Again we have 
VaR(B) =0. When A and B form a portfolio C, the probability that at least one bond will 


default becomes 1—(1—396)(1—396) € 5.994. So VaR(C) 2 $1M » VaR(A)  VaR(B). 


Lack of sub-additivity directly contradicts the intuitive idea that diversification reduces 
risk. So it is a theoretical drawback of VaR. 


(Sub-additivity is one property of a coherent risk measure. A risk measure p(X) is 
considered coherent if the following conditions holds: p(X +Y)< p(X)+ p(Y); 
p(aX)-ap(X) Va>0; p(X)€ p(Y) if X<Y; and p(X +k)=p(X)-& for any 
constant k. It is defined in Coherent Measure of Risk by Artzner, P., et al., Mathematical 
Finance, 9 (3):203—228. Conditional VaR is a coherent risk measure.) 


Duration and convexity 


The duration of a bond is defined as D = oa where P is the price of the bond and y 


2 

is yield to maturity. The convexity of a bond is defined as C= ns Applying 
y 

Taylor's expansion, = =~ —DAy+ > C Ay’. when Ay is small, = = —D Ay. 

For a fixed-rate bond with coupon rate c and time-to-maturity 7: 


'S Stress test is often used as a complement to VaR by estimating the tail risk. 
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TT>Dt cî=D} yT=DV Thectctacy ytscy. 


Another important concept is dollar duration: (aut apy D. Many market 
dy 
lis dP 
participants use a concept called DV01: DV012 ———————, which measures the 
10,000 x dy 


price change when the yield changes by one basis point. For some bond derivatives, 
such as swaps, dollar duration is especially important. A swap may have value P — 0, in 


which case dollar duration is more meaningful than duration. 


When n bonds with values P, i=1,---,, and Durations D, (convexities C.) form a 
portfolio, the duration of the portfolio is the value-weighted average of the durations of 


n P n P n 
the components: D = 2,5D (C- 2,50 ) where P= 5P. The dollar duration of 
i=] i=l 


i=] 


the portfolio is simply the sum of the dollar durations of the components: $D = 5D. 


i=l 


What are the price and duration of an inverse floater with face value $100 and annual 
coupon rate 30%-—3r that matures in 5 years? Assume that the coupons are paid 
semiannually and the current yield curve is flat at 7.5%. 


Solution: The key to solving basic fixed-income problems is cash flow replication. To 
price a fixed-income security with exotic structures, if we can replicate its cash flow 
using a portfolio of fundamental bond types such as fixed-rate coupon bonds (including 
zero-coupon bonds) and floating-rate bonds, no-arbitrage arguments give us the 
following conclusions: 

Price of the exotic security = Price of the replicating portfolio 

Dollar duration of the exotic security = Dollar duration of the replicating portfolio 


To replicate the described inverse floater, we can use a portfolio constructed by shorting 
3 floating rate bonds, which is worth $100 each, and longing 4 fixed-rate bonds with a 
7.5% annual coupon rate, which is worth $100 each as well. The coupon rate of a 
floating-rate bond is adjusted every 0.5 years payable in arrear: the coupon rate paid at 
t+0.5y is determined at t. The cash flows of both positions and the whole portfolio are 


summarized in the following table. It is apparent that the total cash flows of the portfolio 
are the same as the described inverse floater. So the price of the inverse float is the price 
of the replicating portfolio: P... — $100. 
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Short 3 floating- 
-150r —300 — 150r. 
rate bonds i iil 


Long 4 bonds with 
7.5% coupon rate 400 +15 


The dollar duration of the inverse floater is the same as the dollar duration of the 
portfolio as well: $D,,,, 24x $D,,, -3x$D, Since the yield curve is flat, 


inverse oating * 


5,- 7.596 and the floating-rate bond is always worth $103.75 (after the payment of 
$3.75, the price of the floating-rate bond is $100) at year 0.5, and the dollar duration’® is 


d(103.75/(1- y/2 
JM S ADEM icio. ctii 9.3 cedido: 
3 dy (1+ y/ 2) l+y/2 
e. c/2 100 


The price of a fixed-rate bond is P= ) ————— + ——————, 
P 2 vid (1+ y/2)" 


of the bond. So the dollar duration of the fixed-rate bond is 


where T is the maturity 


2T 
$D ied __aP_ l E Perino = 410.64. 
dy lty/2\ 7721+ y/2y (l+y/2) 
SO $D, 5 7 AX $D nixed —3*$D ptoating = 1498 and the duration of the inverse floater is 
LD scie = DD, werse / | un = 14.98. 


Forward and futures 


What's the difference between futures and forwards? If the price of the underlying asset 
is strongly positively correlated with interest rates, and the interest rates are stochastic, 
which one has higher price: futures or forwards? Why? 


Solution: Futures contracts are exchange-traded standardized contracts; forward 


contracts are over-the-counter agreements so they are more flexible. Futures contracts 
are marked-to-market daily; forwards contacts are settled at the end of the contract term. 


6 The initial duration of a floating rate bond is the same as the duration of a six-month zero coupon bond. 
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If the interest rate is deterministic, futures and forwards have the same theoretical price: 
F = Se"*"*. where u represents all the storage costs and y represents dividend yield 


for investment assets, convenience yield for commodities and foreign risk-free interest 
rate for foreign currencies. 


The mark-to-market property of futures makes their values differ from forwards when 
interest rates vary unpredictably (as they do in the real world). As the life of a futures 
contract increases, the differences between forward and futures contracts may become 
significant. If the futures price is positively correlated with the interest rate, the 
increases of the futures price tend to occur the same time when interest rate is high. 
Because of the mark-to-market feature, the investor who longs the futures has an 
immediate profit that can be reinvested at a higher rate. The loss tends to occur when the 
interest rate is low so that it can be financed at a low rate. So a futures contract is more 
valuable than the forward when its value is positively correlated with interest rates and 
the futures price should be higher. 


Interest rate models 


Explain some of the basic interest rate models and their differences. 


Solution: In general, interest rate models can be separated into two categories: short-rate 
models and forward-rate models. The short-rate models describe the evolution of the 
instantaneous interest rate R(t) as stochastic processes, and the forward rate models 
(e.g., the one- or two-factor Heath-Jarrow-Morton model) capture the dynamics of the 
whole forward rate curve. A different classification separates interest rate models into 
arbitrage-free models and equilibrium models. Arbitrage-free models take the current 
term structure—constructed from most liquid bonds—and are arbitrage-free with respect 
to the current market prices of bonds. Equilibrium models, on the other hand, do not 
necessarily match the current term structure. 


Some of the simplest short-rate models are the Vasicek model, the Cox-Ingersoll-Ross 
model, the Ho-Lee model, and the Hull-White model. 


Equilibrium short-rate models 
Vasicek model: dR(t) = a(b — R(t))dt +o dW (t) 


When A(t) » b, the drift rate is negative; when R(t) « b,the drift rate is positive. So the 


Vasicek model has the desirable property of mean-reverting towards long-term average 
b. But with constant volatility, the interest rate has positive probability of being negative, 
which is undesirable. 


Cox-Ingersoll-Ross model: dR(t) = a(b- R(t)) dt +o. R(u) dW (t) 
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The Cox-Ingersoll-Ross model keeps the mean-reversion property of the Vasicek model. 
But the diffusion rate o4 R(u) addresses the drawback of Vasicek model by 
guaranteeing that the short rate 1s positive. 


No-arbitrage short-rate models 


Ho-Lee model: dr = O(t)dt + odz 


The Ho-Lee model is the simplest no-arbitrage short-rate model where O(t) is a time- 
dependent drift. A(t) is adjusted to make the model match the current rate curve. 


Hull-White model: dR(r) = a(b(t) — R(t)) dt + odW (t) 


The Hull-White model has a structure similar to the Vasicek model. The difference is 
that b(t) is a time-dependent variable in the Hull-White model to make it fit the current 


term structure. 
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Chapter 7 Algorithms and Numerical Methods 


Although the percentage of time that a quant spends on programming varies with the job 
function (e.g., quant analyst/researcher versus quant developer) and firm culture, a 
typical quant generally devotes part of his or her time to implementing models through 
programming. Therefore, programming skill test is often an inherent part of the 
quantitative interview. 


To a great extent, the programming problems asked in quantitative interviews are similar 
to those asked in technology interviews. Not surprisingly, many of these problems are 
platform- or language-specific. Although C++ and Java still dominate the market, we’ve 
seen a growing diversification to other programming languages such as Matlab, SAS, S- 
Plus, and R. Since there are many existing books and websites dedicated to technology 
interviews, this chapter will not give a comprehensive review of programming problems. 
Instead, it discusses some algorithm problems and numerical methods that are favorite 
topics of quantitative interviews. 


7.1. Algorithms 


In programming, the analysis of algorithm complexity often uses asymptotic analysis 
that ignores machine-dependent constants and studies the running time T(n) —the 


number of primitive operations such as addition, multiplication, and comparison—as the 
number of inputs n — oo.! 


Three of the most important notations in algorithm complexity are big- O notation, Q 
notation and © notation: 


O( g(n))= { f (n): there exist positive constants c and n, such that 0< f(n) < cg(n) for 
all n2 n, }. Itis the asymptotic upper bound of f(n). 


O(g(n))7 { f(n): there exist positive constants c and n, such that 0 < cg(n) < f(n) for 


all nz n, }. It is the asymptotic lower bound of f(n). 


e( g(n)) = { f(n): there exist positive constants c,, c,, and n, such that 
cg(n) s f(n)€og(n) forall n2 n, }. Itis the asymptotic tight bound of f(n). 


Besides notations, it is also important to explain two concepts in algorithm complexity: 


! If you want to review basic algorithms, I highly recommend "Introduction to Algorithm” by Thomas H. 
Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein. It covers all the theories discussed in 
this section and includes many algorithms frequently appearing in interviews. 
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Worst-case running time W (n): an upper bound on the running time for any n inputs. 
Average-case running time A(n): the expected running time if the n inputs are 


randomly selected. 


For many algorithms, W(n) and A(n) have the same O(g(n)). But as we will discuss 


in some problems, they may well be different and their relative importance often 
depends on the specific problem at hand. 


A problem with n inputs can often be split into a subproblems with n/b inputs in each 
subproblem. This paradigm is commonly called divide-and-conquer. If it takes f(n) 
primitive operations to divide the problem into subproblems and to merge the solutions 
of the subproblems, the running time can be expressed as a recurrence equation 
T(n) =aT(n/b)+ f(n), wherea2 1, b» 1,and f(n)20. 

The master theorem is a valuable tool in finding the tight bound for recurrence 
equation T(n) 2 aT(n/b)4 f(n): If f(n)- O(n ul for some constant €> 0, 


T(n)= O(n" | , since f (n) grows slower than "Ld If f (n) e (n^ log" n) for 
some k 20, T(n) = (1% log"! J , since f(n) and n" ^^ grow at similar rates. If 
f(n)- O(n - for some constant £ » 0, and af(n/b)<cf(n) for some constant 


c «1l, T(n) - G( f (n)), since f(n) grows faster than neo 


Let’s use binary search to show the application of the master theorem. To find an 
element in an array, if the numbers in the array are sorted (a, <a, $--- € a,), we can use 
binary search: The algorithm starts with a,,,. If a,,,,—x, the search stops. If 
a „nj? X, we only need to search a,,---,4,,,._,). If a.) « x, we only need to search 
4,5, 7, 4,. Each time we can reduce the number of elements to search by half after 
making one comparison. So we have a = l|] b = 2, and f(n)-l. Hence, 


f(n)- e (n^ log" n) and the binary search has complexity O(logz7). 


Number swap 
How do you swap two integers, i and j, without using additional storage space? 


Solution: Comparison and swap are the basic operations for many algorithms. The most 
common technique for swap uses a temporary variable, which unfortunately is forbidden 
in this problem since the temporary variable requires additional storage space. A simple 
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mathematic approach is to store the sum of i and j first, then extract i’s value and assign 

it to j and finally assign j’s value to i. The implementation is shown in the following 
2 

code: 


void swap(int &i, int &j) { 


i = i + j; //store the sum of i and j 
j = i- j; //change j to i's value 
i = i - j; //change i to j's value 


} 


An alternative solution uses bitwise XOR (^) function by taking advantage of the fact 
that x^x=0and0^x=x: 


void swap(int &i, int &j){ 


^ 


l £1 J3 
jm OA JS. deo AD UN oS 
1 om^ eed, SOL cg. 303] 


Unique elements 


If you are given a sorted array, can you write some code to extract the unique elements 
from the array? For example, if the array is [1, 1, 3, 3, 3, 5, 5, 5, 9, 9, 9, 9], the unique 
elements should be [1, 3, 5, 9]. 
Solution: Let a be an n-element sorted array with elements a, < a, € --- X a, ,. Whenever 
we encounter a new element a, in the sorted array, its value is different from its 
previous element (a, #a,,). Using this property we can easily extract the unique 
elements. One implementation in C++ is shown as the following function: 
template «class T» vector<T> unique(T a[], int n) { 
vector«T» vec; // vector used to avoid resizing problem 
vec.reserve(n); //reserver to avoid reallocation 
vec.push back (a[0]); 


for(int i=l; i«n; ++i) { 


This chapter uses C++ to demonstrate some implementations. For other problems, the algorithms are 
described using pseudo codes. 
The following is a one-line equivalent function for swapping two integers. It is not recommend, though, as 
it lacks clarity . 

void swap(int &i, int &j) { i-=j=(it=j)-j; }; 
? I should point out that C++ STL has general algorithms for this basic operation: unique and unique copy. 
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if(a[i] != a[i-1]) 
vec.push back(a[1i]); 


) 


return vec; 


Horner's algorithm 
Write an algorithm to compute y= A4, - Ax * 4x * Ax! x A x". 


Solution: A naive approach calculates each component of the polynomial and adds them 
up, which takes O(n^) number of multiplications. We can use Horner's algorithm to 
reduce the number of multiplications to O(n). The algorithm expresses the original 
polynomial as y= ((((4,x + A, X * A, S) x t A, )x +A, )x +A, and sequentially 
calculate B, = 4, B, ,=B x+A_,,°:°:,B, = Bx A. We have y- B, with at most n 
multiplications. 


Moving average 


Given a large array A of length m, can you develop an efficient algorithm to build 
another array containing the n-element moving average of the original array 


( B,,---, B, = NA, B, - (A + A, +--+ 4, )/n, Vi=n,-::,m)? 


i-n4l i-n+2 


Solution: When we calculate the moving average of the next n consecutive numbers, we 
can reuse the previously computed moving average. Just multiply that average by n, 
subtract the first number in that moving average and then add the new number, and you 
have the new sum. Dividing the new sum by 7 yields the new moving average. Here is 
the pseudo-code for calculating the moving average: 


S= A[1] + --- + A[n]; B[n] = S/n; 
for (=n+1 to m) { S = S — A[i-n] + A[i]; B[1] = S/n; } 


Sorting algorithm 


Could you explain three sorting algorithms to sort n distinct values A,,---,A, and 
analyze the complexity of each algorithm? 


Solution: Sorting is a fundamental process that is directly or indirectly implemented in 
many programs. So a variety of sorting algorithms have been developed for different 
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purposes. Here let’s discuss three such algorithms: insertion sort, merge sort and quick 
sort. 


Insertion sort: Insertion sort uses an incremental approach. Assume that we have sorted 
subarray A[1, ..., i-1]. We insert element A, into the appropriate place in A[1, ..., i-1], 


which yields sorted subarray A[1, ..., i]. Starting with i=1 and increases i step by step 
to n, we will have a fully sorted array. For each step, the expected number of 
comparisons is i/2 and the worst-case number of comparisons is i. So we have 


A(n) = of S12 =@(n’) and W(n) = eS = e(n). 
i=l i=l 


Merge sort: Merge sort uses the divide-and-conquer paradigm. It divides the array into 
two subarrays each with n/2 items and sorts each subarray. Unless the subarray is small 
enough (with no more than a few elements), the subarray is again divided for sorting. 
Finally, the sorted subarrays are merged to form a single sorted array. 


The algorithm can be expressed as the following pseudocode: 
mergesort(A, beginindex, endindex) 
if beginindex < endindex 
then centerindex <— (beginindex + endindex)/2 
mergel <- mergesort(A, beginindex, centerindex) 
merge2 <- mergesort(A, centerindex + 1, endindex) 
merge(mergel, merge2) 


The merge of two sorted arrays with n/2 elements each into one array takes ©(n) 
primitive operations. The running time 7(n) follows the following recursive function: 


2T(n/2)+ O(n), if n>1 
T(n) = 

l, ifn=1 
Applying the master theorem to T(n) with a=2,b=2, and f(n)=Q(n), we have 
f(n)= Ofn" log’ n). So T(n) = O(nlogn). For merge sort, A(n) and W (n) are the 
same as T(n). 


Quicksort: Quicksort is another recursive sorting method. It chooses one of the 
elements, A,, from the sequence and compares all other values with it. Those elements 
smaller than A, are put in a subarray to the left of A,; those elements larger than A, are 
put in a subarray to the right of A,. The algorithm is then repeated on both subarrays 
(and any subarrays from them) until all values are sorted. 
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In the worst case, quicksort requires the same number of comparisons as the insertion 
sort. For example, if we always choose the first element in the array (subarray) and 
compare all other elements with it, the worst case happens when 4,,---, A, are already 


sorted. In such cases, one of the subarray is empty and the other has n—1 element. Each 


step only reduces the subarray size by one. Hence, W (n) = aps | = (n). 


i-l 


To estimate the average-case running time, let's assume that the initial ordering is 
random so that each comparison is likely to be any pair of elements chosen from 
A, ,:::, A,. If we suspect that the original sequence of elements has a certain pattern, we 


can always randomly permute the sequence first with complexity O(») as explained in 
the next problem. Let A , and A , be the pth and qth element (1< p «q. n) in the final 
sorted array. There are q — p + 1 numbers between Á , and Á ,- The probability that Á 2 


and A, is compared is the probability that A, is compared with A , before Ay, OF 


A 
q-l 
different subarrays and will not be compared), which happens with probability 


pul? 


is compared with either Á, or A , (otherwise, A, and A, are separated into 


P(p,q)= — (you can again use the symmetry argument to derive this probablity). 
-p+ 
n q-l n q-l 2 
The total expected number of comparison is A(n) = Y. P.q)= >>» 5] 
q=2 p=! 42 prt — Pt l 
= O(n lg n). 


Although theoretically quicksort can be slower than merge sort in the worst cases, it 1s 
often as fast as, 1f not faster than, merge sort. 


Random permutation 


A. If you have a random number generator that can generate random numbers from 
either discrete or continuous uniform distributions, how do you shuffle a deck of 52 
cards so that every permutation is equally likely? 


Solution: A simple algorithm to permute n elements is random permutation by sorting. It 
assigns a random number to each card and then sorts the cards in order of their assigned 
random numbers.^ By symmetry, every possible order (out of m! possible ordered 
sequences) is equally likely. The complexity is determined by the sorting step, so the 


* If we use the continuous uniform distribution, theoretically any two random numbers have zero 
probability of being equal. 
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running time is @(nlogn). For a small n, such as n=52 in a deck of cards, the 
complexity ©(nlogn) is acceptable. For large n, we may want to use a faster algorithm 
known as the Knuth shuffle. For n elements A[1],---, 4[n], the Knuth shuffle uses the 
following loop to generate a random permutation: 


for (71 to n) swap(A[i], A[Random(i, n)]), 


where Random(i, n) is a random number from the discrete uniform distribution between 
i and n. 


The Knuth shuffle has a complexity of O(n) and an intuitive interpretation. In the first 
step, each of the n cards has equal probability of being chosen as the first card since the 
card number is chosen from the discrete uniform distribution between 1 and n; in the 
second step, each of the remaining n —1 cards elements has equal probability of being 
chosen as the second card; and so on. So naturally each ordered sequence has 1/ 7! 
probability. 


B. You have a file consisting of characters. The characters in the file can be read 
sequentially, but the length of the file is unknown. How do you pick a character so that 
every character in the file has equal probability of being chosen? 


Solution: Let's start with picking the first character. If there is a second character, we 
keep the first character with probability 1/2 and replace the pick with the second 
character with probability 1/2. If there is a third character, we keep the pick (from the 
first two characters) with probability 2/3 and replace the pick with the third character 
with probability 1/3. The same process is continued until the final character. In other 
words, let C, be the character that we pick after we have scanned n characters and the 


(n+1)th character exists, the probability of keeping the pick is e and the probability 
n+ 


of switching to the (n+1)th character is — Using simple induction, we can easily 
n4 


prove that each character has 1/m probability of being chosen if there are m characters. 


Search algorithm 


A. Develop an algorithm to find both the minimum and the maximum of » numbers 
using no more than 37/2 comparisons. 


Solution: For an unsorted array of n numbers, it takes n—1 comparisons to identify 
either the minimum or the maximum of the array. However, it takes at most 3n/2 
comparisons to identify both the minimum and the maximum. If we separate the 
elements to 7/2 pairs, compare the elements in each pair and put the smaller one in group 
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A and the larger one in group B. This step takes n/2 comparisons. Since the minimum 
of the whole array must be in group A and the maximum must be in group B, we only 
need to find the minimum in 4 and the maximum in B, either of which takes n/2-1 
comparisons. So the total number of comparisons is at most 3n/2.? 


B. You are given an array of numbers. From the beginning of the array to some position, 
all elements are zero; after that position, all elements are nonzero. If you don't know the 
size of the array, how do you find the position of the first nonzero element? 


Solution: We can start with the 1st element; if it is zero, we check the 2nd element; if the 
2nd element is zero, we check the 4th element... The process is repeated until the ;th 
| 2! 42" e 
step when the 2'th element is nonzero. Then we check the ES element. If it is 
i i-l 


zero, the search range is limited to the elements between the Le M element and 


the 2'th element; otherwise the search range is limited to the elements between the 
i i-l 

2" th element and the LU element... Each time, we cut the range by half. This 

method is basically a binary search. If the first nonzero element is at position n, the 

algorithm complexity is O(log n). 


C. You have a square grid of numbers. The numbers in each row increase from left to 
right. The numbers in each column increase from top to bottom. Design an algorithm to 
find a given number from the grid. What 1s the complexity of your algorithm? 


Solution: Let A be an nxn matrix representing the grid of numbers and x be the number 
we want to find in the grid. Begin the search with the last column from top to bottom: 
A, , 77, A, ,. If the number is found, then stop the search. If A4, , <x, x is not in the grid 


and the search stops as well. If 4,, «x « A 


ln? 


then we know that all the numbers in 


i*l,n? 
rows 1,---,i are less than x and are eliminated as well. Then we search the (i+1)th row 
from right to left. If the number is found in the (i+1)th row, the search stops. If 
A iat 
x. If A, 


1*1, j+! 


>x, x is not in the grid since all the number in rows i+1 and above are larger than 


5X» Á 


i*l.j? 


we eliminate all the numbers in columns j+1,---,n. Then we 


can search along column from 4,,,, towards A, , until we find x (or x does not exist in 


il. j j 


* Slight adjustment needs to be made if n is odd, but the upper bound 37/2 still applies. 
ê i can be 0, which means x < A, ,, in which case we can search the first row from right to left. 
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the grid) or a k that makes 4, , « x « A,,,, and then we search left along the row k +1 
from A 


k+l,j 


its complexity is O(n). 


towards A,,,,... Using this algorithm, the search takes at most 2n steps. So 


Fibonacci numbers 


Consider the following C++ program for producing Fibonacci numbers: 


int Fibonacci(int n) 
{ 
if (n <= 0) 
return 0; 
else if (n==1) 
return 1; 
else 
return Fibonacci (n-1)+Fibonacci(n-2); 


) 


If for some large n, it takes 100 seconds to compute Fibonacci(n), how long will it take 
to compute Fibonacci(m+1), to the nearest second? Is this algorithm efficient? How 
would you calculate Fibonacci numbers? 


Solution: This C++ function uses a rather inefficient recursive method to calculate 
Fibonacci numbers. Fibonacci numbers are defined as the following recurrence: 


F,-0, F =1, F,=F,,+F,4, Vn22 


(ce -(1-V5) 
2" V5 

using induction. From the function, it is clear that 

T(0) =1, TQ) =1, T(n) =T(n-1)+T(n-2) «1. 


F, has closed-formed solution F, = , Which can be easily proven 


So the running time is a proportional to a sequence of Fibonacci numbers as well. For a 
T(n*l) 4541 


large n, 1-45 ” — 6, so 
g ( ) T(n) 2 


. If it takes 100 seconds to compute 


45 «1 


Fibonacci(n), the time to compute Fibonacci(m+1) is T(n+1)* 3 T(n) ~ 162 
seconds.’ 
7 4 N5+1 is called the golden ratio. 


2 
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J5+1 


2 


The recursive algorithm has exponential complexity © | , Which is surely 


inefficient. The reason is that it fails to effectively use the information from Fibonacci 
numbers with smaller n in the Fibonacci number sequence. If we compute K, F, =, F 


n 


in sequence using the definition, the running time has complexity G(). 


An algorithm called recursive squaring can further reduce the complexity to O(log n). 


o [Ea BLP IL [E EF. E FK] [i1 
Since = x and = , we can show that 
Fes) (0 0) pe. En FF} [1 0 


n-l 


Pa L E a , | 1l = 
= using induction. Let A= , we can again apply the divide- 
F F, 1 0 1 0 


The 


, A"? x 47? if n is even 
and-conquer paradigm to calculate A”: A" -| 


AMP? x 407? x A, if n is odd - 
multiplication of two 2x2 matrices has complexity ©(1). So T(n) 2 T(n/2) 4 (I). 
Applying the master theorem, we have T(n) = @(log n). 


Maximum contiguous subarray 


Suppose you have a one-dimensional array A with length n that contains both positive 
and negative numbers. Design an algorithm to find the maximum sum of any contiguous 


J 
subarray Afi, j] of A: Vi, j)= Aix], 1 «is j&n. 


X-l 


Solution: Almost all trading systems need such an algorithm to calculate maximum run- 
up or maximum drawdown of either real trading books or simulated strategies. Therefore 
this is a favorite algorithm question of interviewers, especially interviewers at hedge 
funds and trading desks. 


The most apparent algorithm is an O(n’) algorithm that sequentially calculates the 
V (i, j) 's from scratch using the following equations: 


V (i,i) = A[i] when j =i and Vj) - Y Ap] - VG, j -* ALI when j»i. 


x-i 


As the V (i, j)'s are calculated, we also keep track of the maximum of V (i, j) as well as 
the corresponding subarray indices i and jJ. 
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A more efficient approach uses the divide-and-conquer paradigm. Let’s define 


T(i) => A[x] and T(0)-0, then V(i,j)=T(j)-T(i-l), VISis j<n. Clearly for 
x=! 

any fixed j, when 7(i-1) is minimized, V(i, 7) is maximized. So the maximum 

subarray ending at j is Vax — T(j) - T,,, where 7_,, = min(T(1),---,7T(j-1)). If we keep 


track of and update V „ and T as j increases, we can develop the following O(n) 


algorithm: l B 
T = A[1]; Vaa = A[1]; T_n = min(0,7) 
For j=2 ton 
t T=T+ ALJ); 


If T-T >V then V,,. =T-T. 


min ? 


If T «T. 


min ? 


Return V... ; 


max ? 


then 7 


min 


The following is a corresponding C++ function that returns V_, 
given an array and its length: 


and indices i and j 


X 


double maxSubarray(double A[], int len, int &i, int &j) 
{ 
double T-A[0], Vmax-A[0]; 
double Tmin = min(0.0, T); 
for(int k=1; k<len; ++k) 
{ 
T+=A[k]; 
if (T-Tmin > Vmax) (Vmax-T-Tmin; j=k; } 
if (T<Tmin) {Tmin = T; i = (k+1<j)? (k+1):j;} 
) 
return Vmax; 
) 
Applying it to the following array A, 
double A[]211.0;2.0;-5.0,4.0;-3:.0, 2.0, 6.0, -5.0, -1.0}; 
int i= 0, j =0; 
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double Vmax = maxSubarray(A, sizeof(a)/sizeof(A[1]), i, j); 


will give V „ =9, i=3 and j= 6. So the subarray is [4.0, — 3.0, 2.0, 6.0]. 


7.2. The Power of Two 


There are only 10 kinds of people in the world—those who know binary, and those who 
don't. If you happen to get this joke, you probably know that computers operate using 
the binary (base-2) number system. Instead of decimal digits 0-9, each bit (binary digit) 
has only two possible values: 0 and 1. Binary representation of numbers gives some 
interesting properties that are widely explored in practice and makes it an interesting 
topic to test in interviews. 


Power of 2 
How do you determine whether an integer is a power of 2? 


Solution: Any integer x 2 2" (n2 0) has a single bit (the (n+ 1)th bit from the right) set 


to 1. For example, 8 (2 2^) is expressed as 0---01000. It is also easy to see that 2" —1 
has all the n bits from the right set to 1. For example, 7 is expressed as 0---00111. So 
2" and 2" -1do not share any common bits. As a result, x & (x —1) 2-0, where & isa 
bitwise AND operator, is a simple way to identify whether the integer x is a power of 2. 


Multiplication by 7 
Give a fast way to multiply an integer by 7 without using the multiplication (*) operator? 


Solution: (x << 3) — x, where << is the bit-shift left operator. x << 3 is equivalent to x*8. 
Hence (x << 3) - x is x*7.° 


Probability simulation 


Y ou are given a fair coin. Can you design a simple game using the fair coin so that your 
probability of winning is p, 0 < p «1?* 


* The result could be wrong if «« causes an overflow. 
? Hint: Computer stores binary numbers instead of decimal ones; each digit in a binary number can be 
simulated using a fair coin. 
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Solution: The key to this problem is to realize that p €(0,1) can also be expressed as a 


binary number and each digit of the binary number can be simulated using a fair coin. 
First, we can express the probability p as binary number: 


p-9-D!pP-"-p,- p2" + p,2~ +--+ p,2”, p, € {0,1}, Vial, 2,---, 7. 


Then, we can start tossing the fair coin, and count heads as 1 and tails as 0. Let s, € {0,1} 
be the result of the i-th toss starting from i=1. After each toss, we compare p, with s,. 
If s, « p,, we win and the coin tossing stops. If s, » p,, we lose and the coin tossing 
stops. If s, 2 p,, we continue to toss more coins. Some p values (e.g., 1/3) are infinite 


series when expressed as a binary number (n — oo). In these cases, the probability to 
reach s, + p, is l as i increases. If the sequence is finite, (e.g., 1/4=0.01) and we reach 


the final stage with s, — p, , we lose (e.g., for 1/4, only the sequence 00 will be 


classified as a win; all other three sequences 01, 10 and 11 are classified as a loss). Such 
a simulation will give us probability p of winning. 


Poisonous wine 


You've got 1000 bottles of wines for a birthday party. Twenty hours before the party, 
the winery sent you an urgent message that one bottle of wine was poisoned. You 
happen to have 10 lab mice that can be used to test whether a bottle of wine is poisonous. 
The poison is so strong that any amount will kill a mouse in exactly 18 hours. But before 
the death on the 18th hour, there are no other symptoms. Is there a sure way that you can 
find the poisoned bottle using the 10 mice before the party? 


Solution: If the mice can be tested sequentially to eliminate half of the bottles each time, 
the problem becomes a simple binary search problem. Ten mice can identify the 
poisonous bottle in up to 1024 bottles of wines. Unfortunately, since the symptom won't 
show up until 18 hours later and we only have 20 hours, we cannot sequentially test the 
mice. Nevertheless, the binary search idea still applies. All integers between 1 and 1000 
can be expressed in 10-bit binary format. For example, bottle 1000 can be labeled as 


1111101000 since 100022? + 2° +2’ +2° +2° 4 27. 


Now let mouse | take a sip from every bottle that has a 1 in the first bit (the lowest bit 
on the right); let mouse 2 take a sip from every bottle with a 1 in the second bit; ...; and, 
finally, let mouse 10 take a sip from every bottle with a 1 in the 10th bit (the highest bit). 
Eighteen hours later, if we line up the mice from the highest to the lowest bit and treat a 
live mouse as 0 and a dead mouse as 1, we can easily back track the label of the 
poisonous bottle. For example, if the 6th, 7th, and 9th mice are dead and all others are 
alive, the line-up gives the sequence 0101100000 and the label for the poisonous bottle 


152499 392^ 350. 
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7.3 Numerical Methods 


The prices of many financial instruments do not have closed-form analytical solutions. 
The valuation of these financial instruments relies on a variety of numerical methods. In 
this section, we discuss the application of Monte Carlo simulation and finite difference 
methods. 


Monte Carlo simulation 


Monte Carlo simulation is a method for iteratively evaluating a deterministic model 
using random numbers with appropriate probabilities as inputs. For derivative pricing, it 
simulates a large number of price paths of the underlying assets with probability 
corresponding to the underlying stochastic process (usually under risk-neutral measure), 
calculates the discounted payoff of the derivative for each path, and averages the 
discounted payoffs to yield the derivative price. The validity of Monte Carlo simulation 
relies on the law of large numbers. 


Monte-Carlo simulation can be used to estimate derivative prices if the payoffs only 
depend on the final values of the underlying assets, and it can be adapted to estimate 
prices if the payoffs are path-dependent as well. Nevertheless, it cannot be directly 
applied to American options or any other derivatives with early exercise options. 


A. Explain how you can use Monte Carlo simulation to price a European call option? 


Solution: If we assume that stock price follows a geometric Brownian motion, we can 
simulate possible stock price paths. We can split the time between ¢ and T into N 


T - 
equally-spaced time steps. ' So ALL and t, =t+Atxi, for i-0,1,2,---, N. We 


then simulate the stock price paths under risk-neutral probability using equation 
2 . 

S=5,2° 0 oN Me: where é,'s are IID random variables from standard normal 

distribution. Let's say that we simulate M paths and each one yields a stock price S,.,, 


where k 21,2,:--, M, at maturity date T. 


° For European options, we can simply set N-1. But for more general options, especially the path- 


dependent ones, we want to have small time steps and therefore N should be large. 
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The estimated price of the European call is the present value of the expected payoff, 


M 
> max(S, , — K,0) 


which can be calculated as C = e" ^? &! 


M 


B. How do you generate random variables that follow N(g4,0^) (normal distribution 
with mean 4 and variance o^) if your computer can only generate random variables 
that follow continuous uniform distribution between 0 and 1? 


Solution: This is a great question to test the basic knowledge of random number 
generation, the foundation of Monte Carlo simulation. The solution to this question can 
be dissected to two steps: 


1. Generate random variable of x ~ N(0,1) from uniform random number generator 
using inverse transform method and rejection method. 


2. Scalexto u-- ox to generate the final random variables that follow N(,07). 


The second step is straightforward; the first step deserves some explanations. A popular 
approach to generating random variables is the inverse transform method: For any 
continuous random variable X with cumulative density function F (U = F(X) ), the 


random variable X can be defined as the inverse function of U: X = F(U), OxU <1. 


It is obvious that X =F (U) is a one-to-one function with O<U x1. So any 
continuous random variable can be generated using the following process: 


= Generate a random number z from the standard uniform distribution. 


a Compute the value x such that u= F(x) as the random number from the 
distribution described by F. 


For this model to work, F '(U) must be computable. For standard normal distribution, 
X l 2 

U=F(X)= | ———e""dx. The inverse function has no analytical solution. 
b J2z 

Theoretically, we can come up with the one-to-one mapping of X to U as the numeric 


=x" /2 


solution of ordinary differential equation F'(x) = f(x) =e using numerical 
7A 


integration method such as the Euler method.'' Yet this approach is less efficient than 
the rejection method: 


'! To integrate y = F(x) with first derivative y'= f(x) and a known initial value y, = F(x,), the Euler 
method chooses a small step size h ( h can be positive or negative) to sequentially approximate y values: 
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Some random variables have pdf f(x), but no analytical solution for F~'(U). In these 


cases, we can use a random variable with pdf g(y) and Y 2G (U) to help generate 
random variables with pdf f(x). Assume that M is a constant such that ae <M, Vy. 
404 


We can implement the following acceptance-rejection method: 


e Sampling step: Generate random variable y from g(y) and a random variable v 
from standard uniform distribution [0,1]. 


f(y) 
g( 


e Acceptance/rejection step: If v< , accept x= y; otherwise, repeat the 


sampling step.’ 


An exponential random variable (g(x) = Ae ^ ) with A =1 has cdf u = G(x) -1-e" 
So the inverse function has analytical solution x - -log(1— 4) and a random variable 
with exponential distribution can be conveniently simulated. For standard normal 


-x^/2 


l 
distribution, f(x) = Te : 
20 


f (x) Z 2 n TE «ee ^ 1.32, V0«x«oo 
g(x) Va 74 T 


So we can choose M =1.32 and use the acceptance-rejection method to generate 
x~ N(0,1) random variables and scale them to N(u,0°) random variables. 


C. Can you explain a few variance reduction techniques to improve the efficiency of 
Monte Carlo simulation? 


Solution: Monte Carlo aun: in its basic form, is the mean of IID random variables 


I 


Fosse d rs 3A Since the expected value of each Y, is unbiased, the 


estimator Y is unbiased as well. If Var(Y)=o and we generate IID Y, then 


Var(Y)-60/4 M , where M is the number of simulations. Not surprisingly, Monte Carlo 


F(x, *h) 2 F(x,)+ f(x,))xh, F(x, * 2h) = F(x, - h) - f(x, * h)xh,---. The initial value of the cdf of 
a standard normal can be F(0) = 0.5. 

X< 
2 P(X <x)« | go) ^e dy =M [| fody > F(x) Te | fo» 
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simulation is computationally intensive if o is large. Thousands or even millions of 
simulations are often required to get the desired accuracy. Depending on the specific 
problems, a variety of methods have been applied to reduce variance. 

Antithetic variable: For each series of ¢,'s, calculate its corresponding payoff 
Y(&,:::,£,). Then reverse the sign of all ¢,'s and calculate the corresponding payoff 
Y(-&,-::,—£,). When Y(z,,---,£,) and Y(—é,,::-,-€,) are negatively correlated, the 
variance is reduced. 

Moment matching: Specific samples of the random variable may not match the 
population distribution well. We can draw a large set of samples first and then rescale 


the samples to make the samples' moments (mean and variance are the most commonly 
used) match the desired population moments. 


Control variate: If we want to price a derivative X and there is a closely related 
derivative Y that has an analytical solution, we can generate a series of random numbers 
and use the same random sequences to price both X and Y to yield X and Y. Then X can 
be estimated as X -(Y- Y ). Essentially we use (Y — Y ) to correct the estimation error of 


^ 


X. 


Importance sampling: To estimate the expected value of h(x) from distribution f(x), 
instead of drawing x from distribution f(x), we can draw x from distribution g(x) and 


h(x)f Q9) . 
g(x) 
se s 

g(x) 


use Monte Carlo simulation to estimate expected value of 


E, Ah G9] 7. (AG9 f 69d = [209/09 = E de= E 


ie ADL) 
g(x) 

more efficient estimator. This method is better explained using a deep out-of-the-money 

option as an example. If we directly use risk-neutral f (S,) as the distribution, most of 


has a smaller variance than A(x), then importance sampling can result in a 


the simulated paths will yield h(S,)=0 and as a result the estimation variance will be 


large. If we introduce a distribution g(S,) that has much wider span (fatter tail for S, ), 


POD «nil keen die 
g(x) 


more simulated paths will have positive h(S,). The scaling factor 


estimator unbiased, but the approach will have lower variance. 


? Importance sampling is essentially a variance reduction method using a change of measure. 
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Low-discrepancy sequence: Instead of using random samples, we can generate a 
deterministic sequence of “random variable” that represents the distribution. Such low- 
discrepancy sequences may make the convergence rate 1/ M. 


D. \f there is no closed-form pricing formula for an option, how would you estimate its 
delta and gamma? 


Solution: As we have discussed in problem A, the prices of options with or without 
closed-form pricing formulas can be derived using Monte Carlo simulation. The same 
methods can also be used to estimate delta and gamma by slightly changing the current 
underlying price from S to S+6S, where 6S is a small positive value. Run Monte 
Carlo simulation for all three starting prices S—dOS, S and $+ 6S, we will get their 
corresponding option prices f(S—dOS), f(S) and f(S+0S). 


Estimated delta: A = of = SS + 0S) — f(S—6S) 
os 20S 


(f(S +6S)— f(S))-(/(5) - (S - 65)) 

JS’ 
To reduce variance, it’s often better to use the same random number sequences to 
estimate f(S —óS), f(S) and f(S+6S)."* 


Estimated gamma: I = 


E. How do you use Monte Carlo simulation to estimate 7 ? 


Solution: Estimation of z is a classic example of Monte Carlo simulation. One standard 
method to estimate z is to randomly select points in the unit square (x and y are 
independent uniform random variables between 0 and 1) and determine the ratio of 
points that are within the circle x^ + y^ <1. For simplicity, we focus on the first quadrant. 
As shown in Figure 7.1, any points within the circle satisfy the equation x; + y; <1. The 
percentage of the points within the circle is proportional to its area: 
Number of (x,,y,) within x; +y; <1 l/4Az 1 


— = =—7>7=4p. 
Number of (x,,y,) within the square Ixl 4 


p= 
So we generate a large number of independent (x, y) points, estimate the ratio of the 


points within the circle to the points in the square, and multiply the ratio by 4 to yield an 
estimation of z. Figure 7.1 uses only 1000 points for illustration. With today’s 


^ The method may not work well if the payoff function is not continuous. 
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computing power, we can easily generate millions of (x, y) pairs to estimate z with 
good precision. 1,000 simulations with 1,000,000 (x, y) points each using Matlab took 


less than 1 minute on a laptop and gave an average estimation of z as 3.1416 with 
standard deviation 0.0015. 
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Figure 7.1 A Monte Carlo simulation method to estimate rr 


Finite difference method 


The finite difference method is another popular numerical technique for derivative 
pricing. It numerically solves a differential equation to estimate the price of a derivative 
by discretizing the time and the price of the underlying security. We can convert the 
Black-Scholes-Merton equation, a second order nonlinear partial differential equation, to 
a heat diffusion equation (as we did in Chapter 6). This new equation, expressed as a 
function of t (time to maturity) and x (a function of the price of the underlying security), 
is a general differential equation for derivatives. The difference between various 
derivatives lies in the boundary conditions. By building a grid of x and r and using the 
boundary conditions, we can recursively calculate w at every x and r using finite 
difference methods. 


A. Can you briefly explain finite difference methods? 


Solution: There are several version of finite difference methods used in practice. Let's 
briefly go over the explicit difference method, the implicit difference method and the 
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Crank-Nicolson method. As shown in Figure 7.2, if we divide the range ofr, [0, T ]: into 
N discrete intervals with increment Ar — T/ N and divide the range of x, [x,, x, ], into 


J discrete intervals with increment Ax = (x, — x,)/ J, the time t and the space of x can be 
expressed as a grid of 7,, n -1, ---, N and 3.715599. 


Figure 7.2 Grid of r and x for finite different methods 


The explicit difference method uses the forward difference at time z, and the second- 
Qu u" —u" u” Qu" tu", Qu 
order central difference at x,: —»——_~ = Se —;.. 
AT (Ax) Ox 


Rearranging terms, we can express ut as a linear combination of uw’ 


f+? 


n n .L 
u; and u; : 
n+l 


u^ =au’_,+(1-2a@)u" *au 


n 


jb Where a = At /(Ax)'. Besides, we often have boundary 


conditions u,, 4,, and uf for all n=1,---, N; 7=0,---, J. Combining the boundary 
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conditions and equation 47" = au’_,+(1-2@)u"+au',,, we can estimate all u’'s on 


j+? 
the grid. 
The implicit difference method uses the backward difference at time ¢,,, and the 
l 5 ye 23) ui zr Tu e 
second-order central difference at x, : A gc e SML Riso Mu D xL. 
OT AT (Ax) Ox 


The Crank-Nicolson method uses the central difference at time (f£, +7¢,,,)/2 and the 
second-order central difference at x, : 


(Ax) (Ax) 


n+l n n n n nil n+l i 2 
Ou U; —U, noU U; 2U; tuj. i) 


B. If you are solving a parabolic partial differential equation using the explicit finite 
difference method, is it worse to have too many steps in the time dimension or too many 
steps in the space dimension? 


Solution: The equation for ut in the explicit finite difference method is 


n+l 


u^ =au’_,+(1-2a@)ui +au",,, where a = At/(Ax)’. For the explicit finite difference 


method to * stable, we need to have 1—2a > 0 => At (Ax)? «1/2. So a small At (i.e., 
many time steps) is desirable, but a small Ax (too many space steps) may make 
At /(A^x)^ >1/2 and the results unstable. In that sense, it is worse to have too many steps 


in the space dimension. In contrast, the implicit difference method is always stable and 
convergent. 


j+1? 
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Inclusion-Exclusion Principle, 65 

independence, 73 

induction, 27, 29 

insertion sort, 175 

integration, 36 


193 


integration by parts, 37 outcome, 59 


integration by substitution, 37, 40 partial derivative, 40 

interest rate model, 168 partial differential equations, 146 
intersection, 60 permutation, 65 

inverse floater, 166 Pigeon Hole Principle, 20, 21 
Ito’s lemma, 135 Poison process, 90 

Jensen’s inequality, 140 Poisson process, 90 
jump-diffusion process, 90 polar integral, 41 
L’Hospital’s rule, 36 portfolio optimization, 163 
Lagrange multipliers, 45 positive definite, 56 

law of total expectation, 93, 113 positive semidefinite, 56 
Law of total probability, 73 principle of counting, 64 
linear least squares, 52 Principle of Optimality, 122 
linear regression, 53 probability density function, 41, 86 
logic, 6 probability mass function, 86 
low-discrepancy sequence, 188 probability space, 59 

LU decomposition, 57 product rule, 34, 37 

Markov chain, 105 product rule:, 33 

Markov property, 105, 114 proof by contradiction, 31 
mark-to-market, 168 put-call parity, 138 
martingale, 115 QR decomposition, 52 
master theorem, 172 quicksort, 175 

maximum, 35 quotient rule, 33, 37 
maximum drawdown, 180 random permutation, 176 
merge sort, 175 random variable, 60 
minimum, 35 random walk, 115 
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system equation, 127 variance reduction, 186 
Taylor’s series, 42, 43 Vasicek model, 168 

Theta, 149 vector, 51 

transition graph, 105, 109, 111 Vega, 149 

transition matrix, 105 Volga, 157 

union, 60 Wald’s Equality, 116 
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